Ambari blueprints-overview
Download as PPTX, PDF1 like197 views
This document discusses Hortonworks' Ambari extensibility features of stacks and blueprints. Stacks define services and components and their lifecycles. Blueprints allow defining cluster layouts using stacks. The document provides examples of stack and service definitions as well as blueprint APIs and a single node blueprint definition. However, it notes that some features discussed may still be under development.
![© Hortonworks Inc. 2014: DO NOT SHARE. CONTAINS HORTONWORKS CONFIDENTIAL & PROPRIETARY INFORMATION
Example: Single-Node Definitions
Page 13
{
"configurations" : [
{
”hdfs-site" : {
"dfs.namenode.name.dir" : ”/hadoop/nn"
}
}
],
"host_groups" : [
{
"name" : ”uber-host",
"components" : [
{ "name" : "NAMENODE” },
{ "name" : "SECONDARY_NAMENODE” },
{ "name" : "DATANODE” },
{ "name" : "HDFS_CLIENT” },
{ "name" : "RESOURCEMANAGER” },
{ "name" : "NODEMANAGER” },
{ "name" : "YARN_CLIENT” },
{ "name" : "HISTORYSERVER” },
{ "name" : "MAPREDUCE2_CLIENT” }
],
"cardinality" : "1"
}
],
"Blueprints" : {
"blueprint_name" : "single-node-hdfs-yarn",
"stack_name" : "HDP",
"stack_version" : "2.0"
}
}
{
"blueprint" : "single-node-hdfs-yarn",
"host_groups" :[
{
"name" : ”uber-host",
"hosts" : [
{
"fqdn" : "c6401.ambari.apache.org”
}
]
}
]
}
BLUEPRINT
CLUSTER INSTANCE
Description
• Single-node cluster
• Use HDP 2.0 Stack
• HDFS + YARN + MR2
• Everything on c6401](https://image.slidesharecdn.com/ambari-blueprints-overview-150514023910-lva1-app6891/85/Ambari-blueprints-overview-13-320.jpg)
Ambari blueprints-overview
- 1. © Hortonworks Inc. 2014: DO NOT SHARE. CONTAINS HORTONWORKS CONFIDENTIAL & PROPRIETARY INFORMATION Disclaimer • This document may contain product features and technology directions that are under development or may be under development in the future. • Technical feasibility, market demand, user feedback, and the Apache Software Foundation community development process can all affect timing and final delivery. • This document’s description of these features and technology directions does not represent a contractual commitment from Hortonworks to deliver these features in any generally available product. • Product features and technology directions are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind. 1
- 2. © Hortonworks Inc. 2014: DO NOT SHARE. CONTAINS HORTONWORKS CONFIDENTIAL & PROPRIETARY INFORMATION Ambari Extensibility Blueprints + Stacks May 2014 Page 2
- 3. © Hortonworks Inc. 2014: DO NOT SHARE. CONTAINS HORTONWORKS CONFIDENTIAL & PROPRIETARY INFORMATION Agenda • Ambari Stacks • Ambari Blueprints • Demo Page 3
- 4. © Hortonworks Inc. 2014: DO NOT SHARE. CONTAINS HORTONWORKS CONFIDENTIAL & PROPRIETARY INFORMATION Ambari Stacks Page 4
- 5. © Hortonworks Inc. 2014: DO NOT SHARE. CONTAINS HORTONWORKS CONFIDENTIAL & PROPRIETARY INFORMATION Ambari Stacks • Separates “Stack” from “Ambari” • Makes it easier to bring more than a traditional “Hadoop Stack” under Ambari management • Define a consistent Service lifecycle management interface that can be dynamically extended Page 5
- 6. © Hortonworks Inc. 2014: DO NOT SHARE. CONTAINS HORTONWORKS CONFIDENTIAL & PROPRIETARY INFORMATION Stack Terminology Page 6 Term Definition Examples STACK Defines a set of Services, where to obtain the software packages and how to manage the lifecycle. HDP-2.0, HDP-1.3.3 SERVICE Defines the Components that make-up the Service. HDFS, NAGIOS, YARN COMPONENT The building-blocks of a Service, that adhere to a certain lifecycle. NAMENODE, DATANODE, OOZIE_SERVER CATEGORY The category of Component. MASTER, SLAVE, CLIENT
- 7. © Hortonworks Inc. 2014: DO NOT SHARE. CONTAINS HORTONWORKS CONFIDENTIAL & PROPRIETARY INFORMATION Stack Mechanics • Stacks define Services + Repos –What is in the Stack, and where to get the bits • Each Service has a definition –What Components are part of the Service • Each Service has defined lifecycle commands –start, stop, status, install, configure • Lifecycle is controlled via command scripts • Ability to define “custom” commands Page 7 AMBARI SERVER Stack Command Scripts Service Definitions AMBARI AGENT/S AMBARI AGENT/S AMBARI AGENT/S pythonxml Repos
- 8. © Hortonworks Inc. 2014: DO NOT SHARE. CONTAINS HORTONWORKS CONFIDENTIAL & PROPRIETARY INFORMATION Ambari Blueprints Tech Preview Page 8
- 9. © Hortonworks Inc. 2014: DO NOT SHARE. CONTAINS HORTONWORKS CONFIDENTIAL & PROPRIETARY INFORMATION Ambari Blueprints Page 9 STACK DEFINITION COMPONENT LAYOUT BLUEPRINT BLUEPRINT INSTANTIATE CLUSTER
- 10. © Hortonworks Inc. 2014: DO NOT SHARE. CONTAINS HORTONWORKS CONFIDENTIAL & PROPRIETARY INFORMATION Blueprint Mechanics Page 10 Install, Setup and Start Ambari Server1 Install, Setup and Start Ambari Agents2 Define Blueprint (stack, layout and configurations)3 Define cluster instance4 Instantiate cluster instance and “watch” progress5
- 11. © Hortonworks Inc. 2014: DO NOT SHARE. CONTAINS HORTONWORKS CONFIDENTIAL & PROPRIETARY INFORMATION Key API Calls • GET /blueprints • POST /blueprints/{name} • POST /clusters/{name} • GET /clusters/{name}?format=blueprint Page 11
- 12. © Hortonworks Inc. 2014: DO NOT SHARE. CONTAINS HORTONWORKS CONFIDENTIAL & PROPRIETARY INFORMATION Blueprint API Page 12 BLUEPRINT POST /blueprints/my-blueprint CLUSTER INSTANCE POST /clusters/MyCluster 1 2
- 13. © Hortonworks Inc. 2014: DO NOT SHARE. CONTAINS HORTONWORKS CONFIDENTIAL & PROPRIETARY INFORMATION Example: Single-Node Definitions Page 13 { "configurations" : [ { ”hdfs-site" : { "dfs.namenode.name.dir" : ”/hadoop/nn" } } ], "host_groups" : [ { "name" : ”uber-host", "components" : [ { "name" : "NAMENODE” }, { "name" : "SECONDARY_NAMENODE” }, { "name" : "DATANODE” }, { "name" : "HDFS_CLIENT” }, { "name" : "RESOURCEMANAGER” }, { "name" : "NODEMANAGER” }, { "name" : "YARN_CLIENT” }, { "name" : "HISTORYSERVER” }, { "name" : "MAPREDUCE2_CLIENT” } ], "cardinality" : "1" } ], "Blueprints" : { "blueprint_name" : "single-node-hdfs-yarn", "stack_name" : "HDP", "stack_version" : "2.0" } } { "blueprint" : "single-node-hdfs-yarn", "host_groups" :[ { "name" : ”uber-host", "hosts" : [ { "fqdn" : "c6401.ambari.apache.org” } ] } ] } BLUEPRINT CLUSTER INSTANCE Description • Single-node cluster • Use HDP 2.0 Stack • HDFS + YARN + MR2 • Everything on c6401
- 14. © Hortonworks Inc. 2014: DO NOT SHARE. CONTAINS HORTONWORKS CONFIDENTIAL & PROPRIETARY INFORMATION Status of Blueprints • Basics covered here: https://cwiki.apache.org/confluence/display/AMBARI/Blueprints • “GA” with Ambari 1.6.0 Page 14