Data focused docker clustering

Editor's Notes

• #5: So let’s look at what that means today… take a look at a sample modern application. There’s a huge macro trend towards modern applications being complicated distributed systems, as shown in this example. (And this example doesn’t even included monitoring, logging, caching or lock servers.) These sophisticated apps consists of lots of moving parts - typically, a stateless application tier, here shown on the right, and a data tier, shown on the left, indicated with a database icon. Some parts are horizontally scalable, as indicated by double-boxes. Current platforms do not make it easy for you to move these data services, and certainly not automatically. There’s this hugely important class of data services which (of course) depend upon having a resilient writable file system.
• #6: Containers - and PaaS in general - are great at capturing the stateless (data-free) parts of the applications. When well-designed, according to Heroku’s guidelines known as the 12factor manifesto, the application tier is stateless - which btw was a great business move by Heroku, to avoid having to do the hard work. :) Docker has done a great job of helping people to agree on how to encapsulate their stateless app tiers so that they can run in exactly the same way on your laptop, your staging server, or in production. This is the whole shipping container metaphor that everyone’s super excited about. However, these complex applications depend on stateful services too. In a public IaaS or PaaS, there exist external/outsourced data services, but these don’t exist in a standardised or consistent way in a private PaaS let alone on your developer’s laptop, and that’s a huge problem for devops because it means they can’t guarantee consistency between dev and production. We’re here to change that. You can actually containerise these data services today, but as soon as you do, the data gets locked to the server it’s on, and from our years of experience doing ops we know that’s no good because every server fails eventually. Container platforms today have no answer to the problems of data portability, let alone resilience, failover and versioning, all of which are critically important problems to solve for critical data in challenging production environments. In other words, containers are great for dev environment consistency but it’s not ready yet for ops people who want to use it in production. Docker and the Deis, Flynn crowd are super early, and even the more established players like CloudFoundry and OpenShift don’t have answers to these problems. As Adrian says, “watch the hand-wave”. They only tackle the stateless app tier which is a small component of just about every real application, because real applications have databases, queues and key value stores in them!
• #7: The technology that we bring to the table, backed by 5 years of hard computer science, R&D and 7 patents is how you make that data tier portable too, using resilient distributed storage, as well as how to encapsulate the whole application including the data parts in a self-organising cluster with no single point of failure. This is THE big problem that needs to be solved before Docker can be used to put data and applications together into production.