Apache Ambari - What's New in 1.6.1
Download as PPTX, PDF6 likes2,778 views
This document describes new features and enhancements in Apache Ambari 1.6.1, including new host checks, external database connection testing, stack repository management improvements, and performance optimizations. Key additions are custom JDK checks during host registration, simplified JDBC driver setup, testing of database connections, and managing HDP and HDP-UTILS repositories and base URLs from Ambari Web.
![© Hortonworks Inc. 2014
Background: Current JDK Options
• Ambari Server setup provides three JDK options
• Use options [1] or [2] for Ambari to automatically
download + install the JDK on all hosts
–Requires Internet access
• Use option [3] to specify an already installed JDK
–This is a must when performing a no-Internet / local install
–Ambari expects this JDK to already be installed on ALL hosts
Page 4](https://image.slidesharecdn.com/ambari-whats-new-161-140709091906-phpapp02/85/Apache-Ambari-What-s-New-in-1-6-1-4-320.jpg)
![© Hortonworks Inc. 2014
Custom JDK Host Check
• Problem: when selecting option [3], if a host did not
already have the JDK installed, service and component
starts would fail late in the Cluster Install Wizard process
–Difficult to troubleshoot
• Solution: New “Custom JDK” Host Check performed
during Host Registration
–Confirms the Custom JDK is installed on each host
Page 5](https://image.slidesharecdn.com/ambari-whats-new-161-140709091906-phpapp02/85/Apache-Ambari-What-s-New-in-1-6-1-5-320.jpg)
![© Hortonworks Inc. 2014
ambari.repo Changes
Page 16
The ambari.repo file no longer includes HDP-UTILS, only AMBARI
[root@c6402 ~]# more /etc/yum.repos.d/ambari.repo
[AMBARI.1.6.1-1.x]
name=Ambari 1.x
baseurl=http://public-repo-1.hortonworks.com/ambari/centos6/1.x/updates/1.6.1/
gpgcheck=1
gpgkey=http://public-repo-1.hortonworks.com/ambari/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
1](https://image.slidesharecdn.com/ambari-whats-new-161-140709091906-phpapp02/85/Apache-Ambari-What-s-New-in-1-6-1-16-320.jpg)
Apache Ambari - What's New in 1.6.1
- 1. © Hortonworks Inc. 2014 Apache Ambari 1.6.1 July 2014
- 2. © Hortonworks Inc. 2014 What’s New in 1.6.1 • New Host Checks • External Database Connection Testing • Stack Repository Mgmt Enhancements • Performance Improvements • For a complete list of changes https://issues.apache.org/jira/browse/AMBARI/fixforversion/12326545 Page 2
- 3. © Hortonworks Inc. 2014 New Host Checks Custom JDK Hostname resolution Reverse hostname Page 3
- 4. © Hortonworks Inc. 2014 Background: Current JDK Options • Ambari Server setup provides three JDK options • Use options [1] or [2] for Ambari to automatically download + install the JDK on all hosts –Requires Internet access • Use option [3] to specify an already installed JDK –This is a must when performing a no-Internet / local install –Ambari expects this JDK to already be installed on ALL hosts Page 4
- 5. © Hortonworks Inc. 2014 Custom JDK Host Check • Problem: when selecting option [3], if a host did not already have the JDK installed, service and component starts would fail late in the Cluster Install Wizard process –Difficult to troubleshoot • Solution: New “Custom JDK” Host Check performed during Host Registration –Confirms the Custom JDK is installed on each host Page 5
- 6. © Hortonworks Inc. 2014 Hostname Resolution Check • Checks that each host can resolve hostname for other hosts in the cluster Page 6
- 7. © Hortonworks Inc. 2014 Reverse Hostname Check • On each host, check hostname matches reverse lookup Page 7
- 8. © Hortonworks Inc. 2014 External Database Connections • Simplified JDBC Driver Setup • Ability to “test” Database Connections Page 8
- 9. © Hortonworks Inc. 2014 Background: External Database Drivers Page 9 • Ambari supports using an external database for Hive Metastore and Oozie • Ambari 1.6.0 (or earlier) experience required user to put the JDBC Driver ON the PLANNED host for Hive Metastore or Oozie Server PRIOR to starting cluster install • Potential chances for error – Wrong host – JDBC Driver in wrong location – Miss this step completely
- 10. © Hortonworks Inc. 2014 Simplified JDBC Driver Setup • User provides driver during ambari-server setup… ambari-server setup --jdbc-db=oracle --jdbc-driver=/path/to/odjcb6.jar • During Cluster Install Wizard… –Ambari checks that user provided the driver during setup –Ambari copies the driver to the correct destination host Page 10 AMBARI SERVER Hive Metastore Oozie JDBC Driver JAR
- 11. © Hortonworks Inc. 2014 JDBC Driver Setup Reminder Page 11
- 12. © Hortonworks Inc. 2014 Test Database Connection • Get JDBC driver from Ambari Server • Distribute to target machine (Oozie or Hive Metastore) • Test the database connection (show success/fail) • Popup warning if user proceeds without testing Page 12 1 2 3
- 13. © Hortonworks Inc. 2014 Stack Repository Mgmt HDP and HDP-UTILS managed per Stack Template-driven .repo files Manage Base URLs from Ambari Web Page 13
- 14. © Hortonworks Inc. 2014 Background: Repositories Page 14 AMBARI Repo HDP-UTILS Repo HDP Repo AMBARI HDP STACK Ambari Server and Agent Packages Hadoop Stack Packages Stack Utils LZO, ExtJS, Snappy, Nagios, Ganglia
- 15. © Hortonworks Inc. 2014 Background: .repo Files Page 15 AMBARI Repo HDP-UTILS Repo HDP Repo AMBARI HDP STACK ambari.repo HDP.repo BUT HDP-UTILS is Stack specific
- 16. © Hortonworks Inc. 2014 ambari.repo Changes Page 16 The ambari.repo file no longer includes HDP-UTILS, only AMBARI [root@c6402 ~]# more /etc/yum.repos.d/ambari.repo [AMBARI.1.6.1-1.x] name=Ambari 1.x baseurl=http://public-repo-1.hortonworks.com/ambari/centos6/1.x/updates/1.6.1/ gpgcheck=1 gpgkey=http://public-repo-1.hortonworks.com/ambari/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins enabled=1 priority=1 1
- 17. © Hortonworks Inc. 2014 HDP + HDP-UTILS Stack Repositories Page 17 Prompted to provide HDP and HDP-UTILS Stack repository Base URLs2
- 18. © Hortonworks Inc. 2014 HDP.repo and HDP-UTILS.repo Page 18 Ambari Server Ambari Agent 4) Server sends HDP.repo & HDP-UTILS.repo 2) Reads Stack Base URL values 6) Agent uses HDP.repo HDP-UTILS.repo when installing packages 1) Agent reports 3) Creates HDP.repo & HDP-UTILS.repo ** 5) Agent puts HDP.repo & HDP-UTILS.repo in place ** Generated from Stack template file
- 19. © Hortonworks Inc. 2014 Stack .repo Template File • HDP.repo and HDP-UTILS.repo template file /var/lib/ambari- server/resources/stacks/HDP/2.0.6/hooks/before- INSTALL/templates/repo_suse_rhel.j2 https://github.com/apache/ambari/blob/trunk/ambari- server/src/main/resources/stacks/HDP/2.0.6/hooks/before- INSTALL/templates/repo_suse_rhel.j2 Page 19 Ambari generates the Stack .repo files from a template3
- 20. © Hortonworks Inc. 2014 Managing Repositories Page 20 Manage repository Base URLs from Ambari Web4
- 21. © Hortonworks Inc. 2014 Performance + Scale Page 21
- 22. © Hortonworks Inc. 2014 2,000 Node Clusters • Testing Ambari with 2,000 node clusters • Continued to move Web UI logic to Server • Big focus on Hosts page (filtering, sorting, paging) • Ambari Session at Hadoop Summit 2014 http://www.slideshare.net/Hadoop_Summit/managing-2000-node-cluster-with-ambari http://www.youtube.com/watch?v=sVsHfmZ4s2M Page 22
- 23. © Hortonworks Inc. 2014 Learn More Page 23 Resource Location Apache Ambari Project Page http://ambari.apache.org Ambari Project Wiki https://cwiki.apache.org/confluence/display/AMBARI Ambari Project JIRA https://issues.apache.org/jira/browse/AMBARI