An introduction to Big Data
0 likes1,063 views
This document provides an introduction to big data, including what it is, sources of big data, and how it is used. It discusses key concepts like volume, velocity, variety, and veracity of big data. It also describes the Hadoop ecosystem for distributed storage and processing of large datasets, including components like HDFS, MapReduce, Hive, HBase and ecosystem players like Cloudera and Hortonworks. The document outlines common big data use cases and how organizations are deploying Hadoop solutions in both on-premise and cloud environments.
An introduction to Big Data
- 1. An Introduction to Big Data Concept & Ecosystem
- 2. What is Big Data? Volume • 100s of TBs • PB scale • Too big for traditional transaction processing Velocity • Distributed, Parallel Processing Variety • Structured & unstructured content Veracity • Trustworthiness, Reliability
- 3. Drivers for Big Data Adoption Big Data Adoption Commodity Hardware Support Open Source Ecosystem Web Economy Reduced Storage Costs
- 4. Sources of Big Data Archives Documents Media Business Apps, Data Storage Public Web Social Media Machine/ Sensor Data
- 5. Where it is used? Trends Patterns Predictions
- 6. Usage ScenariosWhatwedo? Activities Conversations Social Media Photographs Videos Transactions WhatbigdataDoes? Text Analysis Speech Analysis Sentiment Analysis Spending Analysis Geographical Analysis
- 7. Working with Big Data Data Source / Ingestion Data Storage Data Processing/ Transformation Data Analysis & Output
- 8. Hadoop Combination of MapReduce engine and HDFS Shift of responsibilities for availability & distribution Brings processing closer to the data
- 9. Hadoop Eco-System Apache Hadoop HBase, Cassandra Hive, Pig Sqoop Mahout MapReduce, HDFS Database Structured Queries RDBMS Connectivity Machine Learning/ Data Mining
- 17. Hive • Started as a sub-project of Hadoop • Now a top-level Apache project • Provides SQL like abstraction layer over MapReduce • Has its own HDFS table file format (and it’s fully schema-bound) • Can also work over Hbase • Acts as a bridge to many BI products which expect tabular data
- 18. Big Data + NoSQL CAP Theorem Consistency AvailabilityPartition Tolerance NoSQL Relational
- 22. Additional Information } To learn more about big data & the eco-system,, get in touch with us. training@forwardsprint.com www.forwardsprint.com Thank you! www.forwardsprint.com