SAP HANA SPS10- Hadoop Integration
2 likes10,954 views
This document discusses new features in SAP HANA SPS 10 for Hadoop and Spark integration, including a native Spark SQL integration using a Spark adapter, Ambari integration with the HANA cockpit for unified administration of HANA and Hadoop nodes, and data lifecycle management between HANA and Hadoop using a relocation agent. It also provides steps for configuring the Spark controller and details the Ambari integration with the HANA cockpit.
SAP HANA SPS10- Hadoop Integration
- 1. 1© 2014 SAP AG or an SAP affiliate company. All rights reserved. SAP HANA SPS 10 - What’s New? Hadoop integration Balaji Krishna, SAP HANA Product Management (Delta from SPS 09 to SPS 10)
- 2. © 2015 SAP SE or an SAP affiliate company. All rights reserved. 3Public New Features for Hadoop/Spark integration • HANA SparkSQL integration using native Spark Adapter Parallel pushing data to Spark Parallel fetching data/result from Spark Enable join- relocation when connecting to Hadoop as Remote Source • Ambari integration with HANA Cockpit Unified UI to administer HANA and Hadoop nodes First step towards a more integrated monitoring • Data Lifecycle Management with Hadoop (Beta)* Hadoop Relocation Agent for Data Lifecycle Manager (DLM) tool
- 3. © 2015 SAP SE or an SAP affiliate company. All rights reserved. 4Public Spark Adapter for HANA HANA Spark SQL Adapter is the key components in HANA – Spark SQL integration. Its main responsibilities are: • Managing connection and communication to a remote Spark SQL Controller, an SAP component deployed on a Hadoop cluster. • Managing access to Spark SQL Controller services. • Translating HANA requests. • Deserialization of data received from Hadoop back HANA. • Browsing Hadoop data structures through the remote source interface. • Enabling the creation of virtual tables through the remote source interface.
- 4. © 2015 SAP SE or an SAP affiliate company. All rights reserved. 5Public Steps to configure Spark controller • Install SAP Spark controller from SMP (sap.hana.spark.controller-1.0-1.noarch.rpm) • Verify new OS user ‘hanaes’ is created • Copy assembly file (spark.controller-1.0.1.jar) to controller lib folder • Perform post install steps like updating hive-site.xml file and adding Hadoop version info • Finally go to Ambari > Resource Manager to check if SAP HANA Spark Controller is running • Detailed instructions are available at SAP Note 2177933
- 5. © 2015 SAP SE or an SAP affiliate company. All rights reserved. 6Public Ambari Integration with HANA Cockpit • New Cockpit dynamic tile to manage Hadoop clusters • Naive integration using redirect to Ambari Web interface • Requires import of cockpit DU package • Add new roles to the user • com.sap.hana.hadoop.cockpit.ambari.data::Administrator • sap.hana.uis.db::SITE_DESIGNER • sap.hana.uis.db::SITE_USER
- 6. © 2015 SAP SE or an SAP affiliate company. All rights reserved. 7Public How to find SAP HANA documentation on this topic? SAP HANA Platform documentation What’s New – Release Notes Modeling Information – SAP HANA Admin Guide Development Information – SAP HANA Developer Guide References – SAP HANA SQL Reference • In addition to this learning material, you find SAP HANA documentation on SAP Help Portal knowledge center at http://help.sap.com/hana_platform. • The knowledge center is structured according to the product lifecycle: installation > security > administration > modeling > development. So you can find e.g. the SAP HANA Developer Guide in the Develepment section and so forth …
- 7. © 2015 SAP SE or an SAP affiliate company. All rights reserved. Thank you Contact information Balaji Krishna SAP HANA Product Management AskSAPHANA@sap.com