5 Popular Choices for NoSQL on a Microsoft Platform

Editor's Notes

• #3: Not going to get too technical today This is just an overview The goal is to show you what's out there, give you an idea of what might appeal to you This is just the start of your journey
• #5: This picture I didn't draw, it was drawn by Nate Lovette Anything I drew, you can tell because it won't look nearly as good as this
• #7: NoSQL is a big umbrella term that encompasses a lot of databases These are 4 of the most popular models, but they aren't the only ones You can see some of these databases are in multiple categories, That's because they support multiple models
• #8: I'm going to be focusing mainly on document databases today They are similar to key/value databases: lot of key-based operations Often a server-side querying mechanism that takes advantages of data being in a known format Like JSON, XML, etc So this is why you hear the term "NoSQL", because there is no SQL involved in interacting with data
• #9: Think of a document database at the most simple model Of a key/value store, where the value is in a known format You write code where you start with a key, and you ask the database to return the document That corresponds to that key. And the same with creating/updating
• #10: Why were these databases created in the first place? Flexibility: schemas are hard to manage Availability: if a machine does down (on purpose or not), we still want to be able to serve customers Speed: the faster we can get the data to a screen, the more likely they are to purchase, less likely to give up, etc Scaling: demand fluctuates, and we need to deal with peaks and valleys When you start from scratch saying 'we need a database that can do X,Y,Z' one of the tradeoffs That allow for all these things is throwing tables and schemas and SQL out the door There's a lot of joking about "web scale" and "/dev/null" and so on. But yes these databases Do need to actually store and be able to retrieve data reliably. Those are table stakes, if you ask me. Sometimes you need tables, constraints, enforced schemas; but you might be surprised at how often you don't. That's a quick rundown of the why/how/history of NoSQL. Hopefully you're in the mindset tonight of "Okay, it might be a useful tool to have in my box. What software should I be looking at specifically? What are the next steps?"
• #12: In this session, I'll be focusing on 5 databases that fit these basic criteria 1 – document databases only, I'm narrowing the focus a bit 2- runs on Microsoft; this session is meant for windows / .net / Microsoft / azure developers 3 – popular by some measure, this is pretty arbitrary
• #13: Let's talk about popularity for a second Db-engines is a site that puts out this monthly ranking It measures a databases popularity by: search engine results, google trends, stack overflow, jobs, linkedin, twitter Mongodb is the most popular nosql database by far Some notable dbs I'm not covering as Cassandra, which is a columnar database, not document I'm omitted Redis, because it's a key/value store, and it's also more of an in-memory cache than it is a database I've also omitted elasticsearch, solr, which I don't really think of as databases, but I guess they are
• #14: I'm omitting DynamoDB because it's AWS exclusive It's not unheard of for a .NET app to use dynamodb, but it's a bit out of the ordinary I'm omitting memcached… explain origins of couchbase via memcached and couchdb Maybe also mention Cloudant
• #15: I'm omitting marklogic, which is a commercial nosql database Omitting Riak and Hazelcast, because they are key/value store I'm not very familiar with Hazelcast, but Riak doesn't run on Windows either CosmosDb is all the way down at #37 (it was 45 the last time I presented) It recently went through a name change, which isn't going to help this score in the long run. I think it's pretty interesting, but it is azure exclusive.
• #16: Starting to get low on the list RethinkDB is notable; the company went out of business but they did open source RethinkDB. It had a bit of a following, so who knows what's in store for that in the long run. Cloudant is based on CouchDb and BigCouch OrientDb is written in Java, it's multi-model including graph and document Realm, Firebase are mobile focused. I'm not going to focus on mobile too much, but Couchbase has a mobile implementation too; and CouchDb has a "reimplementation" in JavaScript called PouchDb, which is geared toward mobile development. RavenDB is pretty low on the list, even though it's been around for quite some time. I wouldn't normally bring it up in a discussion of "popularity", except that I've noticed anecdotally that .NET devs seem to be a lot more aware of it than the rest of the world (for obvious reasons) So I'm doing a bit of cherry picking here
• #17: So these are the 5 that I've picked for this evaluation I'm definitely biased towards my employer
• #18: This is what I want to do with this session
• #19: But I'm going to try as be fair as possible and say as many nice things about each of these that I can If you want to find mean things that people have said, there is no shortage of that on the internet Each of these tools have been created by smart people and are used by great companies to do great things There is no perfect software. Every database is a sum total of flaws, tradeoffs, decisions, and preferences If you have a question like "why should I use couchbase instead of mongo", I can begrudgingly answer in the general, but in reality the answer depends on so many factors I'm not going to cover these tools in details. If you have a question like "what's the best way to index a date field in couchbase to optimize for time series data", this is not the right session. But, I'm happy to talk about anything after the session is over.
• #20: Ward Cunningham's Law "the best way to get the right answer on the internet is not to ask a question; ... it's to post the wrong answer." No way am I an expert in any of these databases, even couchbase
• #22: First criteria is querying All these databases are typically going to provide ways to get or mutate one document at a time But what else do they have beyond that for querying data? The more options the better? Or too many options? Match the option with the use case?
• #23: NoSQL databases were created in a post-web world Where huge amounts of people are using a web site or a mobile app And thus one of the things these databases provide is scalability How well do they scale, how difficult is it to scale? How many steps and decisions are involved? Peer-to-peer is easiest master-slave and replica sets being more difficult
• #24: This may not be a big deal to you, if the database otherwise does what you want it to and does it well We are tech people, we can tinker for a while But I think making software as easy as possible to get started makes our lives much better So I'm going to call out features that I think make it a more pleasant experience "Make common things easy, rare things possible" Keep in the mind the tradeoff here, you may be sacrificing some level of tweaking for ease of use And you may be sacrificing security sometimes if you aren't paying close enough attention
• #25: Many nosql databases make a lot of claims about speed I'm wary of benchmarks So I'm going to focus on architectural decisions that affect speed It's generally bigger, complex data where speed becomes a problem When in doubt do your own benchmarks
• #26: Since we're focusing Microsoft, obviously this is going to lean towards windows and azure But it's nice to know what other options are out there, in case you need to go In a different direction, or use different infrastructure for whatever reason
• #27: Pure technology is not the only thing that matters "The biggest challenges in adopting #NoSQL are usually human rather than technical" - @JudahGabriel Who is doing the support? What are the licenses? Is it open source, does it have a big community? Responsive community? Is it going to be around a year from now? Are they innovating, adding new features I want? Features I may want in the future?
• #29: I do not like this query syntax, It's limited in terms of joins, unions, etc Text search is present, but it's limited compared to elasticsearch, etc .NET SDK follows .NET idioms pretty well, there's a bit of weirdness having to convert objects to BsonDocuments, but it's not that bad Linq provider built into Mongo .NET SDK, but limited due to the underlying query capabilities
• #30: Scaling is possible, you set up multiple types of nodes, configure sharding Replication is a master/slave setup, meaning that a single member of the cluster is the master and is the only one allowed to modify data And that includes between data centers
• #31: Decision fatigue Insecure by default: anonymous admin access
• #32: Indexing is important for querying Mongo has an in-memory option and an on-disk option So that's a basic tradeoff you can make
• #33: Windows / Mac / Linux AWS / Azure / Google support (it is VMs) Docker DBaaS – lot of managed partners like Mongolab, MongoSoup, etc MongoDB Atlas is mongo's own managed DBaaS
• #34: Licensing! It's AGPL, and some enterprises don't like that. In fact, they list it as a possible weakness/threat in their S-1
• #35: CouchDB
• #36: Mongo-inspired query syntax, based on Cloudant Query this is in version 2.x though MapReduce which is great for performance, but it is javascript and doesn't accommodate adhoc queries Mango is "mongo inspired" I don't know if it's mongo compatible or meant to be mongo compatible
• #37: You can setup a cluster relatively easy But you need to run a proxy in front of it, like HAProxy (couchdb recommends) Sharding you need to configure the number of shards per database Replication and conflict management is something that couchdb is good at When removing a node from a cluster you have to make sure to move shards away So there is some manual work involved in managing scale
• #38: Futon is web console in 1.x Fauxton is web console in 2.x Insecure by default: anonymous admin access
• #39: CouchDb's design assumes that caching will be handled by the operating system, by the browser, by a proxy you setup Not by couchdb itself
• #40: Windows / Mac / Linux AWS / Azure / Google support (VMs) Docker (there isn't an official 2.x on docker hub yet) DBaaS – Cloudant
• #41: Cloudant is compatible Couchbase Lite and Sync Gateway are "compatible" with couchdb (version 2 of Couchbase Lite will probably change that) But interop between them is not supported
• #43: I really like N1QL, it's kinda what attracted me to couchbase in the first place I already know how to write SQL, so I can apply that a nosql database Linq2Couchbase is a linq provider that generates N1QL (it's not officially supported yet)
• #44: Multi-master
• #45: Security: with 4.x it was *mostly* secure by default You need to setup a password, but you can create buckets without passwords With 5.x it is completely secure by default
• #46: Explain memcached and couchdb
• #47: Windows / Mac / Linux AWS / Azure / Google support (it is VMs) Docker no DBaaS, managed database (yet)
• #49: Cosmos DB
• #50: There is a SQL language for CosmosDB but it is very limited No intra-document joins, no GROUP BY, no insert/update/delete Cosmos has sprocs, triggers, udfs, but you have to write them in ECMAScript 2015 (JavaScript)
• #51: Azure handles the scaling for you You set request units per second or per minute and cosmos db will scale to handle that Cosmosdb has 5 consistency options, so you can explicitly trade off between strong consistency and eventual consistency, "guarantee" is Microsoft's wording
• #53: "guarantee" is Microsoft's wording
• #54: This is azure only You can run the emulator on windows, the emulator is not meant for production But I'm sure some joker is going to try it The emulator is also available in docker so hypothetically you could deploy that anywhere
• #55: Microsoft support only, of course This is not open source (which used to be implied with Microsoft, but I feel like I have to say that now)
• #58: In Raven 3.x each database is an independent entity, you can setup replication and cooperation, but Oren: "There is a lot of work that you need to do on all the nodes" which "can grow very tedious" Oren on 4.x: "You can bring in additional nodes without having to update any configuration" 4.X is in release candidate now I'm not convinced it's the right database for large scales, but it has some interesting functionality
• #59: Ravendb is on docker, but they also publish a handy Powershell script For getting up and running with docker Which I think is a nice usability touch Secure by default, but you can turn on anonymous admin access
• #60: I think the auto-indexing feature is really intriguing Basically, it will create indexes as you need them And keep them around until they aren't used anymore Sounds great in theory, I've heard in practice that it doesn't quite work out as well as it sounds And you still end up needing to create indexes declaratively I've not seen any bold claims or benchmarks showing raven blowing everyone away in speed Mostly they claim to be a "safe by default" database, so maybe that's the tradeoff they're making They've announced performance improvements in 4.x Raven is built on .NET, some people will claim this hinders performance Just as people will claim databases built on the JVM There may be some truth to this, databases often use low-level operations .NET Core? "RavenDB High Performance " by Brian Ritchie
• #61: Raven 4.x runs on Linux Raven can run on aws/azure in VMs Raven run on windows and 4.x runs in docker RavenHQ is a hosted ravendb provider
• #62: It's agpl Bizspark discounts
• #63: Marten is not a database it's a .NET library that stands between your application and postgresql Postgresql has some really good json support Marten leverages that to treat postgresql as a document database
• #65: Why just one?
• #66: Mobile? look at Couchbase, Sync Gateway, Couchbase Mobile, and maybe CouchDb Querying? Look at Couchbase, CosmosDb, Raven, Marten Cost? Mongo, CouchDb Ops/DevOps proficient? Mongo, CouchDb, Raven Ops/DevOps deficient? CosmosDb, Couchbase, Marten, MongoDb Atlas Hobby? Mongo, CouchDb, Couchbase Resume? Go with the popular one. Speed? Maybe not Raven Transactions? Raven, Marten Security? Be careful with Mongo, CouchDb, not because they are insecure, but because they are insecure by default Integrations? This is huge. Databases generally don't just sit behind one app. They need to integrate with other software. Maybe stick to relational, unless a connector exists.
• #69: We've had Jeremy Miller to talk about Marten, I was a guest host we've had Oren talk about Raven, I was a guest host we've had Kirin to talk about DocumentDb (which is now CosmosDb), I was traveling and couldn't guest host We've had Jonathan Casarrubias to talk about the MEAN stack with mongo)
• #71: Open source apache license for community edition, enterprise edition on a faster release schedule, some advanced features, and support license. Couchbase is software you can run in the cloud on a VM or on your own data center. CosmosDb is a manage cloud service, but there is a emulator you can run locally.
• #72: If you want to play with N1QL (SQL for JSON) you don't even have to install Couchbase first You can do it in browser CosmosDb has one of these too
• #73: Main points of diff: Architecture & Features Architecture Memory first: integrated cache, you don't need to put redis on top of couchbase Master-master: easier scaling, better scaling Auto-sharding: we call vBuckets, you don't have to come up with a sharding scheme, it's done by crc32 Features N1QL: SQL, mongo has a more limited query language and it's not SQL-like Full Text Search: Using the bleve search engine, language aware FTS capabilities built in Mobile & sync: Mongo has nothing like the offline-first and sync capabilities couchbase offers Mongo DOES have a DbaaS cloud provider
• #74: Everything I've shown you today is available in Community edition The only N1QL feature I can think of not in Community is INFER The Enterprise features you probably don't need unless you are Enterprise developer.
• #75: Not yet. We've been talking about it at least as long as I've been with Couchbase. It's partly a technical problem, may need additional features for multi-tenant. It's partly (mostly) a business problem. Would this be worth it? Couchbase IS in the Azure and AWS marketplaces, and there are some wizards to make config easy, but it runs on your VMs. There is no vendor lock in. You can deploy CB to Azure, AWS, GCP, or all three