Building and Managing Reliable Infrastructure with Chef and Chef Delivery
Download as PPTX, PDF5 likes1,075 views
The document discusses Chef and Chef Delivery software for building and managing reliable infrastructure. It provides an overview of Chef's history and products, including Chef for configuration management. Chef Delivery creates a repeatable software delivery pipeline for testing and approving code changes. It treats infrastructure code the same as application code by incorporating versioning, testing, and change management practices. The pipeline standardizes the development stages and allows for customizing activities at each stage.
![COMMON PIPELINE
One
Pipeline
Delivery Pipeline - Shared
union rehearsal delivered
Acceptance Pipelines - Private
review approve deliverChange
Cookbook [A]
review approve deliverChange
Cookbook [B]
review approve deliverChange
Application [A]
review approve deliverChange
Application [B]
U-071982-C
Infrastructure & Applications](https://image.slidesharecdn.com/wallsipexpo2015-chefdelivery-151008132032-lva1-app6891/85/Building-and-Managing-Reliable-Infrastructure-with-Chef-and-Chef-Delivery-17-320.jpg)
![PHASE EXECUTION
log "Running unit"
repo = node['delivery_builder']['repo']
execute “run my junit tests" do
command "mvn test"
cwd repo
end
U-071982-C](https://image.slidesharecdn.com/wallsipexpo2015-chefdelivery-151008132032-lva1-app6891/85/Building-and-Managing-Reliable-Infrastructure-with-Chef-and-Chef-Delivery-23-320.jpg)
![CONFIG.JSON
{
"version": "1",
"build_cookbook": "./.delivery/build_cookbook",
"build_nodes": {
"default": [“name:*buildnode-*”],
"unit": [“name:*-unittest*”],
"lint": [“name:*-linttest*”],
"build": [“name:*-builder*”]
}
}
U-071982-C](https://image.slidesharecdn.com/wallsipexpo2015-chefdelivery-151008132032-lva1-app6891/85/Building-and-Managing-Reliable-Infrastructure-with-Chef-and-Chef-Delivery-27-320.jpg)
Building and Managing Reliable Infrastructure with Chef and Chef Delivery
- 1. BUILDING AND MANAGING RELIABLE INFRASTRUCTURE Mandi Walls, Chef Software IPExpo Europe, October 8, 2015 With Chef and Chef Delivery
- 2. WHO IS CHEF? • Founded about 8 years ago • Seattle-based, with offices in London and San Francisco • Customers include web-natives, retail, financial services • Core product, Chef, is client/server configuration management
- 3. WHOAMI? • Director of Consulting and Customer Success EMEA • Joined Chef in November 2011 • @lnxchk • These slides will be uploaded • http://www.slideshare.net/lnxchk
- 4. CORE CHEF • Sophisticated toolset to produce reliable infrastructure • Tested, versioned, recorded
- 5. THE CODED BUSINESS • Infrastructure as Code – the underlying idea that computing components should be describable in software, versioned, and tested • Expanded to include risk-reduction, value acceleration, and expanded trust
- 6. EASIER TO FIX BUGS BEFORE CODE GETS TO PRODUCTION
- 7. PUSHING MORE INTO PRE-DEPLOY • Testing with ChefSpec, Test Kitchen, Food Critic • Additional system controls via Chef Audit • Don’t get to production and discover that the security rules prevent the app from working • Full 360° view of the application, no assumptions, everything documented in code
- 8. NEED A TOOL THAT MAKES SUBMITTING, APPROVING, AND BUILDING FAST AND EASY
- 9. WHY DELIVERY? • Production of a repeatable pipeline for software delivery • Software isn’t creating value at the end of the build; it creates value after deployment • Application of key concepts to all code, including the code that builds other code
- 10. NOT JUST APPLICATION CODE • We’re now working with infrastructure code, too • It should be auditable and traceable • Incorporate good change management behaviors • Reduce risk
- 11. CHEF DELIVERY
- 12. HOW IT WORKS CLI Web Browser Job Dispatch Push Chef Server erlang Delivery erlang pgSQL DB git SCM Build NodeBuild NodeBuild Nodes U-071982-C
- 13. U-071982-C THE PIPELINE CONCEPT • The build server should reflect modern distributed infrastructures • Preference for smaller, independent, loosely coupled but dependent services to deliver a fully functioning application • Individual software projects ship on their own schedule, but must integrate!
- 14. FAILURE TO SUCCESSFULLY INTEGRATE MEANS THE BUILD SHOULD FAIL BEFORE PRODUCTION
- 15. STABLE PIPELINE Steps Create a new change1 Test Change Locally2 Verification Tests Submit Change3 Review Change4 Build Artifacts Acceptance Tests Approve Change5 Release Process Deliver Change6 manual automated U-071982-C
- 16. THE SHARED PIPELINE • Across various products and teams • Coordination, integration • New releases can trigger refreshes on dependent components
- 17. COMMON PIPELINE One Pipeline Delivery Pipeline - Shared union rehearsal delivered Acceptance Pipelines - Private review approve deliverChange Cookbook [A] review approve deliverChange Cookbook [B] review approve deliverChange Application [A] review approve deliverChange Application [B] U-071982-C Infrastructure & Applications
- 18. UNIFIED SHAPE • The stages are fixed. No change goes to production without flowing through the stages • Stages include specific phases to reduce confusion and increase predictability and stability – these are made up of tasks you’re probably already doing • Your goal is to build software that has value for you, not bikeshed on the pipeline. We did that.
- 19. COMMON PIPELINE Steps manual automated Verify Lint Syntax Unit Build Merge Lint Syntax Unit Quality Security Publish Provision Deploy Smoke Functional Compliance Acceptance Union Provision Deploy Smoke Functional Compliance Rehearsal Provision Deploy Smoke Functional Compliance Delivered Provision Deploy Smoke Functional Compliance Stages customizable Verify Build Acceptance Union Rehearsal Delivered Submit Change3 Review Change4 Approve Change5 Deliver Change6 Chef Delivery Create a new change1 Test Change Locally2 Workstation U-071982-C
- 20. CUSTOMIZE ACTIVITIES • Within the phases, the actions taken are customized to meet the needs of the application • Syntax checking, usability testing, etc, can make use of existing tools • Test nodes can be provisioned on a variety of platforms and environments to meet application needs
- 21. BUILD COOKBOOK ├── recipes ├── default.rb ├── lint.rb ├── syntax.rb ├── unit.rb ├── quality.rb ├── security.rb ├── publish.rb ├── provision.rb ├── deploy.rb ├── smoke.rb └── functional.rb U-071982-C
- 22. BUILD COOKBOOK • Treating the definition of the build process with the same care as the code that is going through it • Allows for versioning of the build definition, tracking of changes, linking of new build definition to new components
- 23. PHASE EXECUTION log "Running unit" repo = node['delivery_builder']['repo'] execute “run my junit tests" do command "mvn test" cwd repo end U-071982-C
- 24. COOKBOOK WORKFLOW U-071982-C Supermarket Chef Server review approve deliverChange Cookbook Node Node Node Node Node Node Node Node Node
- 25. APPLICATION WORKFLOW U-071982-C review approve deliverChange Application Node Node Node Node Node Node Node Node Node Deploy 1 2 3 2 2 3 3 3 3
- 26. U-071982-C CREATE NEW PIPELINES • Project code repository • Build cookbook *versioned!* • Project configuration file *also versioned!*
- 27. CONFIG.JSON { "version": "1", "build_cookbook": "./.delivery/build_cookbook", "build_nodes": { "default": [“name:*buildnode-*”], "unit": [“name:*-unittest*”], "lint": [“name:*-linttest*”], "build": [“name:*-builder*”] } } U-071982-C
- 28. • Delivery gives you a mechanism to create fully-described build process for complex applications • The build process itself benefits from the same version control and software practices the code uses • Prevent bugs and regressions from getting to production with full integration retrospective testing
- 30. UPCOMING EVENTS • Next London Chef Meetup: October 20 • http://www.meetup.com/Chef-Users-London • Chef Cookbook Workflow @ AWS Popup Loft • https://www.chef.io/blog/event/chef-cookbook-workflow- 2/ • Chef Community Summit • November 3-4 in London • https://www.chef.io/summit/london/
- 31. NEW TO CHEF? • https://learn.chef.io/ • Online trainings, in-person classroom training, dates announced regularly
- 32. U-071982-C
Editor's Notes
- #14: Older build and deployment tools reflect the types of systems built in their times – independent projects rather than a family of services working together to create a larger-value application system.
- #17: All production changes ship through this pipeline. Infrastructure changes. Updates to system software. Security fixes. A change is made to the code, it is tested locally on a developer’s workstation for fast response tests with small resource requirements, it’s submitted to the project pipeline, approved, and moves on. We’re in a position now to catch bugs or changes to behavior caused by updates to underlying software. impacts from security updates can be known quickly so system remediation can happen within a shorter window after a vulnerability is announced and fixed by a vendor. Risk is programmatically reduced by employing testing to all code that goes into the Delivery process.
- #18: This means the system is able to help coordinate the flow of change across projects and teams from dev workstation all the way out to Production. Each project has its own acceptance pipeline to run its specific internal tests, whether it is java, ruby, php, javascript, etc. The system enforces a single change-at-a-time moving through each of Union, Rehearsal, and Delivered. These changes have already been internally accepted – they’ve passed their own tests and are provably correct for their own behavior, when they come to the shared pipeline, they are tested against all other dependent services in the cluster, constellation, collection to know that their changes don’t impact the whole. This keeps things stable. If something breaks, you can identify the change that introduced the breakage. When a change is made, it is much more obvious that it is changing the entire system, rather than just a small independent component.