In memory big data management and processing

Seminar By:
Pranav S. Gontalwar

Content
1. Introduction
2. What is Big Data?
3. Characteristics
4. Why Big Data?
5. Processing of Data
6. Structure Of Big Data
7. Sources and Tools
8. Benefits
9. Future of Big Data
10. Conclusion

Introduction
 The first organizations to embrace it were online and
startup firms. Firms like Google, eBay, LinkedIn, and
Facebook were built around big data from the beginning.
 Big Data may well be the next Big thing in IT world. Big
data burst upon the scene in the first decade of the 21st
century.

What is Big data?
 Big Data is similar to small data but bigger in size. But
having data bigger it requires different approaches :
- Techniques,
- Tool and
- Architecture
 Big Data generates value from the storage and processing
of very large quantities of digital information that cannot
be analyzed with traditional computing techniques.

Examples
 Wal-Mart handles more than 1 million customer
transactions every hour.
 Facebook handles 40 billion photos from its user base.

Characteristics
• Volume - Data Quantity
• Velocity - Data Speed
• Variety - Data Types

Volume (Data quantity)
 A typical PC might have had 10 gigabytes of storage in
2000.
 Today, Facebook ingests 500 terabytes of new data every
day.

Velocity (Data Speed)
 High-frequency stock trading algorithms reflect market
changes within microseconds.
 Machine to machine processes exchange data between
billions of devices.
 On-line gaming systems support millions of concurrent
users, each producing multiple inputs per second.

Variety (Data Types)
 Big Data analysis includes different types of data.
 Big Data isn't just numbers, dates, and strings. Big Data is
also geospatial data, 3D data, audio and video, and
unstructured text, including log files and social media.

Processing of Data
 Mapping data to the programming framework.
 Connecting and extracting data from storage.
 Transforming data for processing.

Structure Of Big Data
1.Structured
Most traditional data
sources
2.Semi-structured
Many sources of
big data
3.Unstructured
Video data, audio data

Why Big Data?
 Increase of storage capacities
 Increase of processing power
 Availability of data(different data types)

Tools
 Distributed Servers / Cloud.
 Distributed Storage.
 Distributed Processing
 High-performance schema-free databases

Benefits
• Big data isn’t just a process for storing data in a data
warehouse, It’s about the ability to make better decisions
and take meaningful actions at the right time.
• Improved customer service
• Increased efficiency
• Increases the accuracy of big data

Future of Big Data
 $15 billion on software firms only specializing in data
management and analytics.
 Industry wide global standards, unified communication
protocols, highly enhanced security aspects and
middleware problems are left for future work.

Conclusion
 In memory data management and processing becomes
increasingly interesting for both academia and industry.
 Big data presents opportunity to create unprecedented
business advantages and better service.
 We have focused on the design principles for in-memory
data management and processing, and practical techniques
for designing and implementing efficient and high-
performance in-memory systems.

References
[1]A. Kyrola, G. Blelloch, and C. Guestrin, “Graphchi:
Large-scale graph computation on just a pc,” in Proc. 10th
USENIX Conf. Operating Syst. Des. Implementation, pp.
31–46.
[2]Neo Technology, “Neo4j - the world’s leading graph
database” . [Online]. Available: http://www.neo4j.org/

In memory big data management and processing

More Related Content

What's hot (20)

Similar to In memory big data management and processing (20)

Recently uploaded (20)

In memory big data management and processing