Configuration Management in a Containerized World

Editor's Notes

• #2: I work at Chef as a product manager Which fundamentally means I get to do all the things besides writing the software itself, like business strategy, marketing, making sure we're building the right thing, etc. And I was in charge of the team that built some of the Chef & Docker integration last summer. In a containerized world, is configuration management dead?
• #3: Made the rounds on HackerNews – "CM is dead!" In part there's a bit of new and shiny – many dragons if you read far enough But are there any lessons here?
• #4: And if tech people like nothing better, it's to write articles comparing technology X versus technology Y.
• #5: I honestly never thought I'd have a slide in a presentation with a Chef logo that included a Puppet logo. James is awesome though, and he wrote an article for PuppetLabs asking whether CM and containers can co-exist.
• #6: You don't need runtime state management anymore – which is why people say "docker wins, CM is dead"
• #7: So apparently this is why everyone's rejoicing over configuration management and how it's going to get run over by the container bus… er, ship. I truly believe that yes, CM is not going away, and it's not going to be killed off by containers. So we all have jobs still! Yay!
• #8: - But in order to fully embrace and love containers, traditional CM is, for sure, going to have to change – and change quite dramatically.
• #9: First I need to make sure everyone understands the benefits of containerization and Docker specifically. Who has actually *used* Docker for a real use case? Development is fine (Maybe call on one or two individuals to talk about what they liked about it.)
• #10: Near-instant productivity: Workstation setup is easy Kit of parts in registry Reduces needless "infrastructure knob twiddling" just to be productive Developing is a lot like shipping - Actually, it's a lot like why JVMs are such a great target platform too Developers are nearly-instantly productive Experience of shipping is similar to developing Portable artifact Container artifact is the key concept This is where other technologies like lxc, zones, etc. have fallen down – these were technologies primarily optimized for ops worldview, instead of devs worldview The Good Optimized for developer workflow: Makes developing software really fast Reduces needless "twiddling infrastructure knobs" to ship code The Bad Lots of operational tooling still missing Service discovery, fleet management, resource allocation Build tools are easy until they're not (more on that later)
• #11: This last point is probably the most compelling reason for containers So it's not surprising that there's a land grab happening over it – you can expect more this year.
• #12: Developers are the ones who made are making containers (in whatever format) successful – not operations people. FreeBSD jails, Solaris zones, even LXC – not usable by developers, and not shareable artifacts.
• #13: So a developer's job is to make software artifacts as quickly as possible, and ship them as quickly and as frequently as possible. It doesn't matter whether that artifact is a Java WAR/EAR… Or a container… It's the same fundamental process.
• #14: Seamless build management for containers If possible, ability to use the same infracode across containers/non-containers Want to not distinguish "application" code from "infrastructure" code – it's all just code to enable customer features/value. Seamless build management for containers Provide an experience for use of declarative CM to build containers that is easier than existing tooling Easier than shell scripting Shell scripts are quick but can become painful to maintain Lots of duplication, one-offs Make shell->CM onramp much lower Good job Ansible so far
• #15: None of this involves writing program features It's just the ceremony necessary to get something produced to create value Story of wife formerly a Java developer., etc. Anything that eases a developer's pain in any of this makes their life easier
• #16: - Containers aren't perfect. I outlined some of the ways in which they're great, above, but they have some gaps that I'll talk to next. Always be wary of folks postulating that a "simple" thing is going to replace something "complex", as though simplicity was the ultimate end goal of everything. Simplicity is great, but not a end-goal in itself. Hear of people replacing one CM system for another because they didn't understand the code – it was "too complex". Well 6 months later they've just got spaghetti CM code for that new system b/c they had to build all that complexity in that they didn't understand
• #17: Dockerfiles are a great way to get started but ultimately it is is just a shell script. How do you version it? Sprawl of Dockerfiles No reusable components No way to analyze them, validate them, etc. Also not a great communication tool
• #18: How do folks validate their containers actually meet some criteria? I have no idea. How do folks validate and inspect their running containers for some state? For compliance? For GHOST/POODLE/whatever? This is Chef's audit mode (not the final syntax). I would love to see us extend this to containers as well. Maybe right now the "rebuild" cost is so low for containers that people don't care? But we also didn't just build WAR/EAR files without some external integration testing, right?
• #19: Again, I don't know (aside from publishing images to an internal docker registry) people do this today, but lots of folks are trying to muscle in on this turf AWS CodeDeploy Etc. Lots of different ways to express this using CM as well.
• #20: Run… well we're already pretty good at that part. But… what folks are missing in both the traditional and containerized world in CM is fleet management – crossing the machine boundary.
• #21: Extend CM concepts to cross the machine boundary into managing entire fleets, independent of underlying runtime – should be able to mix-and-match (database on metal, webservers a mix of Azure and EC2 if you wanted to) More on this at 12:00-12:40 in this room – John Keiser will demonstrate these concepts
• #22: Whole Product Solution - CM systems need to become more what Geoffrey Moore calls the "whole product solution"
• #23: Talk in here about the chasm 80% of IT out there uses NO CM, and why? Because I don't think there's a compelling enough business driver from across a broad spectrum, including development.
• #24: Virtualization Remember how virtualization started off as a desktop tool? It migrated to widespread adoption because VMWare seized the day and provided management tools (Vsphere, VCAC, Player, etc.) and an entire ecosystem Arguably, Xen lost the hypervisor battle because they weren't able to provide a whole product solution. They should have won and instead were acquired by Citrix.
• #25: Java Remember how bad Java was when it first started? Primary use case was stupid applets and things like that It became widely adopted when there was an ecosystem: good servlet containers, debugging tools, IDEs
• #26: Java Remember how bad Java was when it first started? Primary use case was stupid applets and things like that It became widely adopted when there was an ecosystem: good servlet containers, debugging tools, IDEs
• #27: Why do I care so much about CM "crossing the chasm"? Not because I want to make $ off my Chef shares (I mean I do, but that's not the primary driver) Not because I fear CM will be "destroyed" by containerization
• #28: Of course it's only the most narcissistic speaker that references his or her previous talks in the current one, right? A few months ago I gave a talk entitled "Devops Against Inhumanity – go and Google it if you're interested; I can give you the headline here The crux is, call it whatever you want, but the "devops" way of working is to lower everyone's stress level, work towards a mutual goal, etc. Dev and Ops have different motivations but they don't have to fight! Good tools reinforce good culture and vice-versa
• #29: - It's one thing to throw WAR files over the wall from dev to ops – terrible, but manageable, you may have worked in such an environment before
• #30: But throwing entire machine images? That's a whole other ball game Unlimited list of things that can go wrong Security? Compliance? We risk losing the goodwill we've built in the "devops" movement where both parties are collaborating Probably why ops is terrified of containerization and is pushing back
• #31: Containers are powerful & they're generally here to stay The format & implementation will change greatly this year & next Configuration management needs to broaden its horizons to remain relevant Development drives technology adoption, but an ops function is still important – no technology is widely adopted when only one party's interests are accommodated (in the long run) So think of CM beyond just file/package/service basics, and create a more whole product solution We've come too far and made too much cultural progress to let that go to waste.