No sql (1)
- 1. 1 No SQL DATABASE MADE BY CLASS 9 GUIDE BY SHAILENDRA KUMAR GUPTA SIR MADE BY :- MUSKAN CHOUHAN :- AANYA JAIN :- PRANJAL SHRIVASTAVA
- 2. 2 Agenda Some history What is NoSQL CAP Theorem What is lost Types of NoSQL Data Model Frameworks Demo Wrapup
- 3. 3 Other ways to scale RDBMS Database Multi-Master replication insert only, not upDates/Deletes no Joins, thereby reDucing query tiMe – this involves De-norMalizing Data in-MeMory Databases
- 4. 4 What is NoSQL? Stands for Not Only SQL Class of non-relational data storage systems Usually do not require a fixed table schema nor do they use the concept of joins All NoSQL offerings relax one or more of the ACID properties (will talk about the CAP theorem)
- 5. 5 How did we get here? Explosion of social media sites (Facebook, Twitter) with large data needs Rise of cloud-based solutions such as Amazon S3 (simple storage solution) Just as moving to dynamically-typed languages (Ruby/Groovy), a shift to dynamically-typed data with frequent schema changes Open-source community
- 6. 6 Dynamo and BigTable – Three major papers were the seeds of the NoSQL movement • BigTable (Google) • Dynamo (Amazon) – Gossip protocol (discovery and error detection) – Distributed key-value data store – Eventual consistency • CAP Theorem (discuss in a sec ..)
- 7. 7 CAP Theorem Three properties of a system: consistency, availability and partitions You can have at most two of these three properties for any shared-data system To scale out, you have to partition. That leaves either consistency or availability to choose from – In almost all cases, you would choose availability over consistency
- 8. 8 Availability Traditionally, thought of as the server/process available five 9’s (99.999 %). However, for large node system, at almost any point in time there’s a good chance that a node is either down or there is a network disruption among the nodes. – Want a system that is resilient in the face of network disruption
- 9. 9 What kinds of NoSQL NoSQL solutions fall into two major areas: – Key/Value or ‘the big hash table’. • Amazon S3 (Dynamo) • Voldemort • Scalaris – Schema-less which comes in multiple flavors, column-based, document-based or graph-based. • Cassandra (column-based) • CouchDB (document-based) • Neo4J (graph-based) • HBase (column-based)
- 10. 10 Key/Value Pros: – very fast – very scalable – simple model – able to distribute horizontally Cons: - many data structures (objects) can't be easily modeled as key value pairs
- 11. 11 Schema-Less Pros: - Schema-less data model is richer than key/value pairs - eventual consistency - many are distributed - still provide excellent performance and scalability Cons: - typically no ACID transactions or joins
- 12. 12 Common Advantages Cheap, easy to implement (open source) Data are replicated to multiple nodes (therefore identical and fault-tolerant) and can be partitioned – Down nodes easily replaced – No single point of failure Easy to distribute Don't require a schema Can scale up and down Relax the data consistency requirement (CAP)
- 13. 13 What am I giving up? joins group by orderby ACIDtransactions SQLas a sometimes frustrating but still powerful query language easy integration with otherapplications that support SQL
- 14. 14 Cassandra Originally developed at Facebook Follows the BigTable data model: column-oriented Uses the Dynamo Eventual Consistency model Written in Java Open-sourced and exists within the Apache family Uses Apache Thrift as it’s API
- 15. 15 Thrift Created at Facebook along with Cassandra Is a cross-language, service-generation framework Binary Protocol (like Google Protocol Buffers) Compiles to: C++, Java, PHP, Ruby, Erlang, Perl, ...
- 16. 16 Typical NoSQL API Basic APIaccess: – get(key) -- Extract the value given a key – put(key, value) -- Create orupdate the value given its key – delete(key) -- Remove the key and its associated value – execute(key, operation, parameters) -- Invoke an operation to the value (given its key) which is a special data structure (e.g. List, Set, Map .... etc).
- 17. 17 Some Statistics Facebook Search MySQL > 50 GB Data – Writes Average : ~300 ms – Reads Average : ~350 ms Rewritten with Cassandra > 50 GB Data – Writes Average : 0.12 ms – Reads Average : 15 ms
Editor's Notes
- #4: -> The multi-master replication system is responsible for propagating data modifications made by each member to the rest of the group, and resolving any conflicts that might arise between concurrent changes made by different members. -> For INSERT-only, data is versioned upon update. -> Data is never DELETED, only inactivated. -> JOINs are expensive with large volumes and don’t work across partitions. -> Denormalization leads to even larger databases, reduces query time. -> Consistency is the responsibility of the application. -> In-memory databases have not caught on mainstream and regular RDBMS are more disk-intensive that memory-intensive. Vendors looking to fix this.
- #5: -> NoSQL was a term coined by Eric Evans. He states that ‘… but the whole point of seeking alternatives is that you need to solve a problem that relational databases are a bad fit for. … -> This is why people are continually interpreting nosql to be anti-RDBMS, it's the only rational conclusion when the only thing some of these projects share in common is that they are not relational databases.’ -> Emil Elfrem stated it is not a ‘NO’ SQL but more of a ‘Not Only” SQL.
- #7: -> BigTable: http://labs.google.com/papers/bigtable.html -> Dynamo: http://www.allthingsdistributed.com/2007/10/amazons_dynamo.html and http://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf -> Amazon and consistency * http://www.allthingsdistributed.com/2010/02 * http://www.allthingsdistributed.com/2008/12
- #8: -> Proposed by Eric Brewer (talk on Principles of Distributed Computing July 2000). -> Partitionability: divide nodes into small groups that can see other groups, but they can't see everyone.-> Consistency: write a value and then you read the value you get the same value back. In a partitioned system there are windows where that's not true.-> Availability: may not always be able to write or read. The system will say you can't write because it wants to keep the system consistent.-> To scale you have to partition, so you are left with choosing either high consistency or high availability for a particular system. You must find the right overlap of availability and consistency. -> Choose a specific approach based on the needs of the service.-> For the checkout process you always want to honor requests to add items to a shopping cart because it's revenue producing. In this case you choose high availability. Errors are hidden from the customer and sorted out later. -> When a customer submits an order you favor consistency because several services--credit card processing, shipping and handling, reporting— are simultaneously accessing the data.
- #10: -> Not an exhaustive list, just some of the more well-known. -> HBase is the data storage solution for Hadoop. -> Graph: is a network database that uses edges and nodes to represent and store data. -> Document: views are stored as rows which are kept sorted by key. Can adapt to variations in document structure.
- #11: -> http://chariotsolutions.blogspot.com/2010/01/why-you-need-nosql-in-your-toolbox.html
- #13: -> As the data is written, the latest version is on at least one node. The data is then versioned/replicated to other nodes within the system. -> Eventually, the same version is on all nodes.
- #14: -> No JDBC -> Data integrity at the application layer
- #16: -> http://incubator.apache.org/thrift/static/thrift-20070401.pdf -> http://incubator.apache.org/thrift/ -> Thrift also created by Facebook engineers and donated to Apache
- #17: -> http://horicky.blogspot.com/2009/11/nosql-patterns.html
- #18: -> http://static.last.fm/johan/nosql-20090611/cassandra_nosql.ppt