Manage Add-On Services with Apache Ambari

1 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Manage Add-on
Services in
Apache Ambari
Jayush Luniya
Dataworks Summit Sydney, 2017

Speakers
Apache Ambari PMC
Engineering Manager @ Hortonworks
jluniya@apache.org
Jayush Luniya

Agenda
 Ambari Overview
 Management Packs – Mpacks v1
 Add-On Services – Primer
 vNext Architecture – Ambari 3.x
 Mpack Based Operations Model – Mpacks v2
 Summary

Ambari Overview

Apache Ambari
Ambari Server
Provision | Manage | Monitor
Ambari REST API
Ambari Stacks
Apache Ambari is a 100% open source
platform for provisioning, managing and
monitoring Apache Hadoop clusters
Other Operations Tools
AmbariAgents

Apache Ambari Themes
• Deliver the core operational capabilities to provision,
manage and monitor Hadoop clusters at scale.
Operate
Hadoop at
Scale
• Robust API for integration with existing enterprise
systems
Integrate
with the
Enterprise
• Provide extensible platform for Customers, Partners
and the Community (Stacks, Views)
Extend for
the
Ecosystem

Ambari Architecture

Extensibility Features
• No vendor lock-in with Ambari
• Customize a stack for customer specific environments
• Add new Services (ISV or otherwise) beyond vendor stack
Stacks
• Use Ambari for automating cluster installations
• Share best practices on layout and cluster configuration
Blueprints
• Extend and customize the Ambari Web UI
• Add new capabilities, customize existing capabilities
Views

Lifecycle Management
Deploy
Configure
Monitor
Secure
Upgrade
Extend

Management Pack – V1

Ambari Management Packs
Motivation

 Decouple stack releases from Ambari core release
 Delivery artifact to release stacks and services
 Release vehicle for add-on services
 Tarballs with metadata of how to install the mpack
 EPIC: AMBARI-14854
 Ambari Wiki
 Release: 2.4.0.0

Add-On Services

Add-on Service
 Mpack Definition
– Specify services, stacks in mpack
– Format: mpack.json
 Service Definition
– Specify components, dependencies, configs
– Format: metainfo.xml
 Lifecycle Management
– Scripts for install/start/stop/upgrade commands
– Format: python scripts
 Role Command Order
– Dependency order for start/stop commands
– Format: role_command_order.json
 Service Repo
– Where to install the service binaries from?
– Format: repoinfo.xml
 Configurations
– Key, value, description, allow empty, password, etc.
– Format: <config-type>.xml
Starter Pack

Add-on Service
 Enhanced UI
– Enhanced UI configuration and dashboard layouts
– Format: theme.json, widgets.json
 Service Advisor
– Recommend/validate configurations
– Format: service-advisor.py
 Alerts
– Defines periodic checks to perform and alert on
– Format: alerts.json
 Metrics
– Defines metrics to be sent from your service to Ambari
– Format: metrics.json
 Kerberos
– Defines service principals, keytab & rules for configuration changes
– Format: kerberos.json
 Service Upgrade Pack
– Steps to perform to upgrade the service (express, rolling)
– Format: upgrades/*.xml
Starter Pack

Mpack Definition
Mpack.json
{
"name": "my-ambari-mpack",
"version": “1.0.0.0",
"description": "Ambari Management Pack for MyService-1 & MyService-2",
"prerequisites": {
"min-ambari-version": "2.5.0.0",
"min-stack-versions": [
{
"stack_name": "HDP",
"stack_version": "2.5.0"
}
]
},
"artifacts": [
{
"name": "my-service-definitions",
"type" : "service-definitions",
"source_dir" : "common-services"
}
],

Mpack Definition
Mpack.json
{
"name" : "my-addon-services",
"type" : "stack-addon-service-definitions",
"source_dir": "addon-services",
"service_versions_map": [
{
"service_name" : "MyService-1",
"service_version" : "2.1.1",
"applicable_stacks" : [
{
"stack_name" : "HDP",
"stack_version" : "2.5"
}
]
},
{
"service_name" : "MyService-2",
"service_version" : "1.0.1",
"applicable_stacks" : [
{
"stack_name" : "HDP",
"stack_version" : "2.6"
}
]
}

Service Definition
Metainfo.xml
<service>
<name>SAMPLESRV</name>
<displayName>New Sample Service</displayName>
<comment>A New Sample Service</comment>
<version>1.0.0</version>
<components>
<component>
<name>SAMPLESRV_MASTER</name>
<displayName>Sample Srv Master</displayName>
<category>MASTER</category>
<cardinality>1</cardinality>
<commandScript>
<script>scripts/master.py</script>
<scriptType>PYTHON</scriptType>
<timeout>600</timeout>
</commandScript>
</component>
<component>
<name>SAMPLESRV_SLAVE</name>
<displayName>Sample Slave</displayName>
<category>SLAVE</category>
<cardinality>1+</cardinality>
...

Service Definition
Metainfo.xml
...
<commandScript>
<script>scripts/slave.py</script>
</commandScript>
<customCommands>
<customCommand>
<name>MY_CUSTOM_COMMAND</name>
<commandScript>
<script>scripts/my_custom_command.py</script>
<timeout>1200</timeout>
</commandScript>
</customCommand>
<customCommands>
<dependencies>
<dependency>
<name>HDFS/HDFS_CLIENT</name>
<scope>cluster | host</scope>
<auto-deploy>
<enabled>true | false</enabled>
</auto-deploy>
</dependency>
<dependencies>

Service Definition
Metainfo.xml
...
</component>
</components>
<requiredServices>
<service>HDFS</service>
</requiredServices>
<configuration-dependencies>
<config-type>service-env</config-type>
<config-type>service-site</config-type>
</configuration-dependencies>
<osSpecifics>
<osSpecific>
<osFamily>any</osFamily>
<packages>
<package>
<name>rpm_apt_pkg_name</name>
</package>
</packages>
</osSpecific>
</osSpecifics>
</service>
</services>
</metainfo>

import sys
from resource_management import Script
class Master(Script):
def install(self, env):
print 'Install the Sample Srv Master'
def stop(self, env):
print 'Stop the Sample Srv Master'
def start(self, env):
print 'Start the Sample Srv Master'
def status(self, env):
print 'Status of the Sample Srv Master'
def configure(self, env):
print 'Configure the Sample Srv Master'
if __name__ == "__main__":
Master().execute()
Python Scripts

Role Command Order
{
"_comment" : "Record format:",
"_comment" : "blockedRole-blockedCommand: [blockerRole1-
blockerCommand1, blockerRole2-blockerCommand2, ...]",
"general_deps" : {
"_comment" : “Dependencies for SAMPLESRV",
"SAMPLESRV_SLAVE-START": ["SAMPLESRV_MASTER-START"],
"SAMPLESRV_SLAVE-STOP": ["SAMPLESRV_MASTER-STOP"]
}
}

Enhanced Configurations
 Too many configurations
– Which ones are important?
 Too easy to mess up
– What are valid/reasonable values?
– What are the units?
– Ok, what about dependencies?
 Gets harder with combinations of services, host assignments,
enabled features, CPU/RAM/disks, etc
– Any recommendations? What am I doing wrong?
 Solution:
– Enhanced Configurations
– Stack/Service Advisor
Hadoop Configuration Challenges

Enhanced Configurations - UI
Customizable layout
- Tabs
- Sections
- Sub-sections
- Simple grid layout
(Advanced Tab
contains remaining
configurations)
New Widgets
Sliders
- Recommended
- Minimum
- Maximum
- Increment Step
Combos
- Enumerated values
Toggles
- Binary options
Spinners
- Splits value into multiple
controls.
- Time in milliseconds split
into days, hours,
minutes.
Lists
- Enumerated values
- Single select
- Multi select

Service Advisor
 Replaces monolithic stack advisors
 Defined at service level
 Config recommendations and validations
 Scenarios
– Kerberos Enabled/Disabled
– HTTPS Enabled/Disabled
– Zookeeper Quorum
– Memory Settings
– High Availability Enabled/Disabled

Alerting Framework
Alert Type Description Thresholds (units)
WEB Connects to a Web URL. Alert status is
based on the HTTP response code
Response Code (n/a)
Connection Timeout
(seconds)
PORT Connects to a port. Alert status is based
on response time
Response (seconds)
METRIC Checks the value of a service metric.
Units vary, based on the metric being
checked
Metric Value (units vary)
Connection Timeout
(seconds)
AGGREGATE Aggregates the status for another alert % Affected (percentage)
SCRIPT Executes a script to handle the alert
check
Varies
SERVER Executes a server-side runnable class to
handle the alert check
Varies

Ambari Metrics System
 Metric Sinks – HDFS, YARN, HBase, Storm etc.
 Metric Monitors – lightweight daemon for system metrics
 Collector – API daemon + HBase (embedded / distributed)
 Phoenix schema designed for fast reads
 Grafana support from version 2.2.2
Ambari
Collector API
Grafana
Phoenix
HDP
Services
System
MONITOR
S
SINKS
Metrics Collector

Kerberos
 Ambari manages Kerberos principals and keytabs
 Works with existing MIT KDC or Active Directory
 Once kerberized, handles
– Adding host components
– Adding hosts
– Adding services
– Moving components

vNext Architecture

Limitations
One-One Relationships
Monolithic
 Stacks are monolithic
 Entire cluster upgrade
 Ambari upgrades for refreshing stack definitions
Ambari Cluster
Cluster Stack
Stack
Service
Version
Cluster
Service
Instance
Host
Component
Instance
Ambari
Hosting
Platform

Vision
 Replace monolithic stacks
 Modular upgrades
 Decouple Ambari core release
 Independent release cycles
 Refactor at service level
 3rd party onboarding platform
 Multiple Clusters
 Multiple Stacks
 Multiple Services
 Multiple Service Versions
 Multiple Host Components
 Multiple Hosting Platforms
Modular Multi Everything

Cluster2
Cluster1
Ambari Hosting Platform
v.Next Architecture
ZK1 HDFS YARN HBASE
Core SG
NIFIZK2 KAFKA
Stream SG
Mpack Repository
HDP
3.0.0
HDF
3.1.0
HDF
3.2.0
Ambari
BROKER-1
BROKER-2
BROKER-3
HBASE2
Yapp SG
YarnApp
1.0.0
Yarn Hosting Platform
HMaster
RServer-1
RServer-2

Cluster2
Cluster1
Ambari Hosting Platform
v.Next Architecture
ZK1 HDFS YARN HBASE
Core SG
NIFIZK2 KAFKA
Stream SG
Mpack Repository
HDP
3.0.0
HDF
3.1.0
HDF
3.2.0
Ambari
BROKER-1
BROKER-2
BROKER-3
HBASE2
Yapp SG
YarnApp
1.0.0
Yarn Hosting Platform
HMaster
RServer-1
RServer-2
Modular
Multi-Everything

Mpack Based Operations
Model

New Concepts - Delivery
 Deployment artifact
 Individual ownership of services
 Release container for services
 Examples:
o HDFS-3.0.0-packlet
o SPARK-2.0.0-packlet
 Distribution artifact
 Composed of multiple packlets
 Modular release & upgrades
 Examples:
o HDP-3.0.0-mpack
o HDF-3.1.0-mpack
Packlets Management Packs (Mpacks)
Mpack Repository
 Catalog of management packs
 Compatibility between mpacks
 Examples:
o Hortonworks Mpack Repository

Management Packs – V2
 Decouple Ambari Core release
 Replacement for monolithic stacks
 Composition of verified combination of packlets
 Support dynamic installation
 Mpacks released and upgraded independently
Release Management

Management Packs – V2
 Support multiple mpack repositories
 Search and discover management packs
 Compatibility between management packs
 Recommendations for mpack bundles
Add-on Mpacks
 First class support for 3rd party add-on services
 Provide a third party onboarding platform
 Independently upgradable services
Mpack Repositories

Upgrades
 Modular upgrades
 Ability to upgrade subset of services in a cluster
 Patch services without affecting other parts of the cluster
 Performance improvements
Goals
Mpack Upgrades
 Upgrade packs defined at mpack level
 Upgrade subset of services in a cluster that belong to a mpack
 Download new version of a mpack and kickoff an upgrade

Upgrades
 Upgrade mechanism for hotfix releases
 Selectively patch components without restarting other services
 Purely Version Definition File (VDF) driven; no configuration changes
 Does not require a new management pack; uses same upgrade pack
 Applied patches can be reverted; no finalization step
Patch Upgrades
Service Upgrades
 VDF driven if no configuration changes needed
 Mpack driven for more complicated service upgrades
 No revert capability (similar to current cluster upgrades)

Summary

Key Takeaways
 Stack Based Operations Model
 Refactoring at service level
 Mpacks v1 provided delivery vehicle
for add-on services
 Architecture limitations remain
 Mpack Based Operations Model
 Re-architect from ground up
 First class support for add-on services
 Modular deployments & upgrades
Current Future
A
M P

Resources
 https://cwiki.apache.org/confluence/display/AMBARI/Stacks+and+Services
 https://cwiki.apache.org/confluence/display/AMBARI/Custom+Services
 https://cwiki.apache.org/confluence/display/AMBARI/Management+Packs
 https://cwiki.apache.org/confluence/display/AMBARI/Enhanced+Configs
 https://cwiki.apache.org/confluence/display/AMBARI/Enhanced+Service+Dashboard
 https://cwiki.apache.org/confluence/display/AMBARI/Stack+Defined+Metrics
 https://cwiki.apache.org/confluence/display/AMBARI/Alerts
 https://cwiki.apache.org/confluence/display/AMBARI/Quick+Links
 https://cwiki.apache.org/confluence/display/AMBARI/Automated+Kerberizaton
 https://cwiki.apache.org/confluence/display/AMBARI/Service+Upgrade
 https://cwiki.apache.org/confluence/display/AMBARI/Role+Command+Order

Apache JIRAs - vNext
 AMBARI-14714 – Multi Everything Architecture
Umbrella Apache EPIC
 AMBARI-19621 – Mpack Based Operations Model
 AMBARI-20463 – Multi Service Instances
 AMBARI-20465 – Multi Host Component Instances
 AMBARI-17353 – Yarn Hosted Services
 AMBARI-12556 – Patch Upgrades
 AMBARI-20466 – Multi Cluster
 AMBARI-20435 - Swagger Integration
Other Apache EPICs
Release Timelines
 Features will be incrementally delivered in Apache Ambari 3.x releases.
Join the Apache
Ambari Community

Thank You

Manage Add-On Services with Apache Ambari

More Related Content

What's hot (20)

Similar to Manage Add-On Services with Apache Ambari (20)

More from DataWorks Summit (20)

Recently uploaded (20)

Manage Add-On Services with Apache Ambari

Editor's Notes