The Life of a Hadoop Administrator, with and without SmartSense
5 likes1,715 views
The document details the experiences of a Hadoop administrator, Alex, who implements Hortonworks SmartSense, enhancing system performance through recommendations and proactive support from Hortonworks engineers. Key interactions involve resolving issues related to HDFS and optimizing configurations, demonstrating the effectiveness of SmartSense. Without SmartSense, troubleshooting becomes arduous, but with it, Alex enjoys a smoother operational experience.
The Life of a Hadoop Administrator, with and without SmartSense
- 1. The Life of a Hadoop Administrator, with and without Hortonworks SmartSense With SmartSense 3 hours later. 1 hour later. 8 hours later. 2 days later. 1 week later. Hello, thank you for calling Hortonworks Support. This is Chris. How can I help you? Ok, do you have SmartSense installed? OK. Please send us HBASE logs from all 150 servers. Logs arrive, Chris calls Parker Chris calls Steve They need to increase their max transfer threads from 4x to 16x Thanks, Steve! Set the max transfer threads from 4k to 16k and then restart HDFS and HBASE. Alex is a Hadoop Administrator at a Fortune 1000 company. Chris is a Hortonworks support engineer. Parker is a Hortonworks HBASE engineer. Steve is a Hortonworks HDFS engineer. Found half the cluster had transparent pages enabled, causing performance issues. Hello Alex. We have new SmartSense recommendations for you. Oh great. I’ll log into the support portal and take a look. Thank you! Wow. This is awesome! Increase datanode Xcievers Disable transparent huge pages Hortonworks is working for me while I sleep. I love my job! HDFS upgrade not finalized HDFS outage 1 week later due to incomplete upgrade. Without SmartSense "I took a look and it's definitely an HDFS issue. Please send us HDFS logs from all 150 servers. And you want to scrub them for IPs and FQDN's this time. Trouble Shooting Purgatory Repeats Itself Trouble Shooting Purgatory Avoided