Fuel, Puppet and OpenStack

Copyright © 2014 Mirantis, Inc. All rights reserved
www.mirantis.com / fuel-infra.org
A Technical Overview
Fuel, Puppet
and OpenStack
March 2, 2015

Introductions
Christopher Aedo — Product Architect
Christopher is an IT veteran for consulting, design and technology companies. He
is also an outspoken public advocate for OpenStack, cloud computing, software
defined networking and software defined storage. He often speaks at OpenStack
and Open Source related conferences around the world.
About Mirantis
Mirantis is the number one pure play OpenStack Company. We deliver all the
technology, integration, training and support required for companies to succeed
with production-grade open source cloud. More customers rely on Mirantis than
any other company to scale out OpenStack without the compromises of vendor
lock-in. Our bench of 400+ open source infrastructure experts helped make us one
of top 5 contributors to OpenStack’s upstream codebase.

Agenda
● Deploying OpenStack
● Fuel Overview & Architecture
● Beginning, challenges, Improvements
● Where Fuel is Headed
● Q&A

Deploying OpenStack

● DevStack
● Deployment tools
● Fuel, Crowbar, Foreman, Helion
● Config/Orchestration
● Puppet, Chef, Ansible, Salt
● Distro-based tools
● Packstack, Juju, RHEL OSP
Deploying OpenStack is easy!

What is Fuel?
● OpenStack deployment and management tool
● Open source, growing community
● GUI and CLI driven
● Visit http://fuel-infra.org to learn more and download

Fuel

http://fuel-infra.org
Fuel Overview & Architecture

Fuel Components: Nailgun

● Written in Python
● Exposes internal data via REST API
● Concentrates up to 90% of orchestration logic
● Transforms user input (UI, CLI) into deployment data
(JSON)
● Keeps tasks status and all information in DB
● Uses SQLAlchemy, Alembic for DB access and migrations
Fuel Components: Nailgun

Fuel Components: Astute
image deployer

● Astute consumes messages sent by Nailgun
● Astute and its agents are written in Ruby
● Astute uses MCollective RPC
● Has bindings to provisioning (cobble or Ironic-based)
● Tracks progress of deployment, provisioning
● Supports operations (adding/removing nodes)
● Small codebase in comparison with Nailgun
Fuel Components: Astute

Fuel Components: OSTF (Health Check)

● Health Check
● Network Checker
● DHCP
● VLANs
● Diagnostic Snapshot
● Logs collection & calculation of operations progress
Fuel: Additional Functions

Fuel Architecture
image deployer

Beginning, Challenges,
Improvements

● Using Puppet Master
● issues w/agent signing key and deploying a node
● catalogs grew too large
● Big catalog, long compilation times
● 5+ minutes with <10 nodes
● time increase between linear and exponential as nodes added
● sending results back to master taking too long
● nginx, separate DB, other optimizations not helping much
Early days and challenges

● Very complex catalogs
● We pull in 1501 resoures and 150 are classes for the controller
role
● Minor modifications cause order to change
● Sequence errors show up
● Loops we thought we’d accounted for pop up
● 20% goes to fixing this stuff

● Drop the master
● Rsync the manifests
● Run puppet-apply!
● Wait, what about orchestration?
● had astute.yaml as data entry point
● already using it for cobbler, extend to puppet!
Improvements

● Hiera didn’t exist yet, so we extended Astute
● $::fuel_settings all over the place
● Worked but not perfect
● Lacks merge/replace elements
● No eyes on issues here outside Fuel devs
● As of v6.1 Fuel uses Hiera
● All puppet manifests have easy access to namespaced data
● Helps with granular deployments
● Soon will use it for populating diff for patches
Improvements

Where Fuel Is Headed

● Granular deployments
● breaking controller role into discrete tasks
● resolves most of the sequence/dependency issues
● improves Fuel plug-in functionality
● New graphing system (task-lib)
● feeds astute
● will eventually move to Mistral
● allows for more complicated catalogs of manifests
● (which then allows use of other peoples composite layers)
What’s coming up

What’s coming up

● Upstream manifests
● forked because we couldn’t keep up
● pulling in upstream, deferring changes with an upstream source
until there’s a commit upstream
● still behind (from 0 to 4 months), but catching up
● by EOY will be on tip of upstream, plan to provide CI testing
What’s coming up

● Server spec (or equivalent) CI integration tests
● tests in the manifests can confirm we end with requested state
● feed in configuration data source, ensure resulting configuration
matches
● few upstream libraries providing this (mostly just using rspec
tests)
● want these tests upstream for gating in the community
What’s coming up

(we’re hiring, ask me about working at Mirantis!)
caedo@mirantis.com - twitter/IRC: @docaedo
Thank you

Fuel, Puppet and OpenStack

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Fuel, Puppet and OpenStack (20)

More from aedocw (6)

Recently uploaded (20)

Fuel, Puppet and OpenStack