Hadoop in the Cloud
Download as PPTX, PDF6 likes724 views
This document discusses Hadoop in the cloud and provides examples of how different cloud providers offer Hadoop-based services. It outlines appliance-based offerings from Oracle, Greenplum, and NetApp before discussing specific Hadoop cloud services from Google App Engine, Amazon Web Services, Windows Azure, and IBM SmartCloud. Each service is briefly described in 1-2 sentences. The document concludes with a quote about an elephant's faithfulness.
Hadoop in the Cloud
- 1. The Elephant in the Room Jim O’Neil Developer Evangelist, Microsoft jim.oneil@microsoft.com @jimoneil
- 2. DIY Roll your own Hadoop cluster…. welcome to DevOps “Isotope” Pallet
- 3. Appliances Oracle Big Data Appliance – 18 server / 12 core each / 40Gb Infiniband – Partnering with Cloudera on the distribution Greenplum HD Data Computing Appliance – 18 nodes, 12 core each – Straight up Apache Hadoop NetApp Open Solution for Hadoop – Storage arrays only (E2660 and FAS2040) – Partnership with Cloudera
- 4. The Elephant in the Cloud Jim O’Neil Developer Evangelist, Microsoft jim.oneil@microsoft.com @jimoneil
- 5. Cloud: a Notional Definition Private Cloud Deployment Models Hybrid Cloud Community Cloud Public Cloud Essential Characteristics Infrastructure as a Service Platform as a Service Broad network Rapid Elasticity access Software as Resource Pooling a Service On-demand self-service Measured service
- 6. Hadoop in the Cloud Google App Engine appengine-mapreduce API (not really Hadoop) Amazon Web Services 66 Public AMIs (including Cloudera) Elastic Map Reduce Windows Azure Hadoop on Azure IBM SmartCloud Infosphere BigInsights
- 7. Google App Engine MapreducePipeline Class Experimental! Mapreduce is an experimental, innovative, and rapidly changing new feature for App Engine. Unfortunately, being on the bleeding edge means that we may make backwards-incompatible changes to Mapreduce. We will inform the community when this feature is no longer experimental.
- 8. Amazon EMR u
- 9. Windows Azure http://HadoopOnAzure.com Currently in Customer Technology Preview Partnership with Hortonworks Windows updates to Apache JavaScript framework Hive ODBC connector
- 10. IBM SmartCloud InfoSphere BigInsights IBM distribution of Hadoop (0.20.2) Jaql query language BigSheets BigInsight Scheduler “Hadoop ecosystem” Hive, Avro, Hbase, Pig, Oozie, Flume
- 11. I meant what I said, and I said what I meant. An elephant's faithful, one hundred percent. Jim O’Neil Developer Evangelist, Microsoft jim.oneil@microsoft.com @jimoneil