Nosql intro
5 likes2,180 views
This document discusses the future of data storage and the rise of NoSQL databases. It notes that while SQL databases have dominated for decades, their suitability is cracking due to limitations in scaling and integration. NoSQL databases are designed to run on clusters across many machines, have flexible schemas, and are open source. They allow for embracing large scale and reducing development drag. However, relational databases are still relevant for some use cases. The future is one of "polyglot persistence" using the best data storage technology for each application's needs.
![so this means we can
Reduce Development Drag Embrace Large Scale
A lot of effort in application development is tied up in The large scale clusters that we can support with
working with relational databases. Although Object/ NoSQL databases allow us to store larger datasets
Relational Mapping frameworks have eased the load, the (people are talking about petabytes these days) to
database is still a significant source of developer hours. process large amounts of analytic data.
Often we can reduce this effort by choosing an alternative
database that’s more suited to the problem domain. Alternative data models also allow us to carry out many
tasks more efficiently, allowing us to tackle problems
We often come across projects who are using relational that we would have balked at when using only relational
databases because they are the default, not because they databases
are the best choice for the job. Often they are paying a
cost, in developer time and execution performance, for
features they do not use.
Danish Health Care
Centralized record of drug
prescriptions. Currently held in
MySQL databases, but concerned
about scale for both response time
and availability. Migrated data to
DNC Riak. [more...]
Searching 300 Million voters
information for 1 person with
addresses, emails, phones is
tough with a relational data store.
McLaren
Guardian MongoDB was used to store the
documents about the person. Streaming of telemetric data into
New functionality uses Mongo rather [more...] MongoDB for later analysis. Orders
than relational DB. They found of magnitude faster than relational
Mongo’s document data model (SQL Server). [more...]
significantly easier to interact with for
5 their kind of application. [more...]](https://image.slidesharecdn.com/nosql-intro-120310091335-phpapp01/85/Nosql-intro-5-320.jpg)
Nosql intro
- 1. The future is: NoSQL Databases Polyglot Persistence Martin Fowler a note on the future of data storage in the enterprise, written primarily for those involved in the management of application development. Pramod Sadalage © Martin Fowler and Pramod Sadalage Rendered: February 8, 2012 11:26
- 2. SQL has Ruled for two decades Store persistent data Application Integration Storing large amounts of data Many applications in an enterprise on disk, while allowing need to share information. By applications to grab the bits getting all applications to use the they need through queries database, we ensure all these applications have consistent, up-to- date data Mostly Standard The relational model is widely used and understood. Interaction with the database is done with SQL, which is a (mostly) standard Concurrency Control language. This degree of Many users access the same standardization is enough to keep information at the same time. things familiar so people don’t Reporting Handling this concurrency is need to learn new things SQL’s simple data model and difficult to program, so databases standardization has made it a provide transactions to help foundation for many reporting ensure consistent interaction. tools All this supported by Big Database Vendors and the separation of the DBA profession. 2
- 3. but SQL’s dominance is cracking Relational databases are designed But it’s cheaper and more effective to run on a single machine, so to to scale horizontally by buying lots of scale, you need buy a bigger machines. machine SQL SQL The machines in these large clusters are Google and Amazon were both early individually unreliable, but the overall cluster adopters of large clusters, and both keeps working even as machines die - so the eschewed relational databases. overall cluster is reliable. The “cloud” is exactly this kind of cluster, which Google Bigtable means relational databases don’t play well with the cloud. Amazon Dynamo The rise of web services provides an effective alternative to shared databases for application integration, making it easier for different Their efforts have been a large inspiration applications to choose their own data storage. to the NoSQL community 3
- 4. so now we have NoSQL databases examples include There is no standard definition of what NoSQL means. The term began with a workshop organized in 2009, but there is much argument about what databases can truly be called NoSQL. But while there is no formal definition, there are some common characteristics of NoSQL databases they don’t use the relational data model, and thus don’t use the SQL language they tend to be designed to run on a cluster they tend to be Open Source they don’t have a fixed schema, allowing We should also remember Google’s you to store any data in any record Bigtable and Amazon’s SimpleDB. While these are tied to their host’s cloud service, they certainly fit the general operating characteristics 4
- 5. so this means we can Reduce Development Drag Embrace Large Scale A lot of effort in application development is tied up in The large scale clusters that we can support with working with relational databases. Although Object/ NoSQL databases allow us to store larger datasets Relational Mapping frameworks have eased the load, the (people are talking about petabytes these days) to database is still a significant source of developer hours. process large amounts of analytic data. Often we can reduce this effort by choosing an alternative database that’s more suited to the problem domain. Alternative data models also allow us to carry out many tasks more efficiently, allowing us to tackle problems We often come across projects who are using relational that we would have balked at when using only relational databases because they are the default, not because they databases are the best choice for the job. Often they are paying a cost, in developer time and execution performance, for features they do not use. Danish Health Care Centralized record of drug prescriptions. Currently held in MySQL databases, but concerned about scale for both response time and availability. Migrated data to DNC Riak. [more...] Searching 300 Million voters information for 1 person with addresses, emails, phones is tough with a relational data store. McLaren Guardian MongoDB was used to store the documents about the person. Streaming of telemetric data into New functionality uses Mongo rather [more...] MongoDB for later analysis. Orders than relational DB. They found of magnitude faster than relational Mongo’s document data model (SQL Server). [more...] significantly easier to interact with for 5 their kind of application. [more...]
- 6. but this does not mean relational is dead the relational model is still ACID transactions relevant In order to run effectively on a The tabular model is suitable for cluster, most NoSQL databases many kinds of data, particularly have limited transactional when you need to pick apart data capability. Often this is enough… and re-assemble it in different but not always. ways for different purposes. Tools Familiarity The long dominance of SQL NoSQL systems are still new, so means that many tools have been people aren’t familiar with using written to work with SQL them. So we shouldn’t be using databases. Tooling for alternative them on utility projects where datastores is much more limited. their benefits would have less impact. 6
- 7. this leads us to a world of Polyglot Persistence using multiple data storage technologies, chosen based upon the way data is being used by individual applications. Why store binary images in relational database, when there are better storage systems? Polyglot persistence will occur over the enterprise as different applications use different data storage technologies. It will also occur within a single application as different parts of an application’s data store have different access characteristics. http://martinfowler.com/bliki/PolyglotPersistence.html 7
- 8. what might Polyglot Persistence look like? Needs high availability across Rapidly traverse links Rapid access for reads Needs transactional multiple locations. Can merge between friends, product and writes. No need to updates. Tabular inconsistent writes purchases, and ratings be durable structure fits data Speculative Retailers Web Application User sessions Financial Data Shopping Cart Recomendations Redis RDBMS Riak Neo4J This is a very hypothetical example, we would not make technology Product Catalog Reporting Analytics User activity logs recommendations without more MongoDB RDBMS Cassandra Cassandra contextual information Lots of reads, infrequent writes. Products make Large-scale analytics on natural aggregate large cluster High volume of writes on multiple nodes SQL interfaces well with reporting tools 8
- 9. polyglot persistence provides lots of new opportunities for enterprises i.e. problems Decisions Organizational Change We have to decide what data How will our data groups react to storage technology to use, rather this new technology? than just go with relational Immaturity Dealing with Eventual Consistency NoSQL tools are still young, and Paradigm full of the rough edges that new tools have. How will different stakeholders in the enterprise data deal with data that Furthermore since we don’t have could be stale and how do you much experience with them, we enforce rules to sync data across don’t know how to use them well, systems what the good patterns are, and what gotchas are lying in wait. 9
- 10. what kinds of projects are candidates for polyglot persistence? If you need to get to market quickly, then you need to maximize productivity of your development team. If appropriate, polyglot persistence can remove significant rapid time to market drag. Strategic and and/or data intensive Most software projects are utility Data intensiveness can come in projects, i.e. they aren’t central to various forms the competitive advantage of the lots of data company. Utility projects should not high availability take on the risk and staffing demands that polyglot persistence lots of traffic: reads or writes brings as the potential benefits are complex data relationships not there. Any of these may suggest non- relational storage, but its the exact nature of the data interaction that will suggest the best of the many 10 alternatives.
- 11. for more information… On the Web We are both active writers on our websites. Martin writes at http://martinfowler.com and Pramod at http://www.sadalage.com/. Forthcoming Book We are currently working on a introductory book to NoSQL databases, to be titled: NoSQL Distilled (see http://martinfowler.com/bliki/ NosqlDistilled.html) Consulting and Delivery ThoughtWorks has carried out several projects delivering production systems using NoSQL technologies. To see if NoSQL is a good fit for your needs, and how we can help your delivery, contact your local ThoughtWorks office, which you can find at http://thoughtworks.com 11