The Berkshelf Way
19 likes26,918 views
The document outlines the Berkshelf tool, which is a command-line application for managing Chef cookbooks similar to package managers like Ruby's gem and Python's pip. It describes how to create new cookbooks, manage dependencies with berksfiles, and handle uploads to a Chef server, as well as the importance of metadata in defining cookbook attributes and dependencies. Additionally, it emphasizes the best practices for writing well-encapsulated cookbooks and optimizing workflows with tools like Vagrant and Berkshelf integration.
![STUCK? ASK FOR HELP!
$ berks cookbook –h
Usage:
berks cookbook NAME
Options:
[--foodcritic] # Creates a Thorfile with Foodcritic support to lint test your cookbook
[--chef-minitest] # Creates chef-minitest support files and directories, adds minitest-handler …
[--scmversion] # Creates a Thorfile with SCMVersion support to manage versions for continuous
integration
-L, [--license=LICENSE] # License for cookbook (apachev2, gplv2, gplv3, mit, reserved)
# Default: reserved
-m, [--maintainer=MAINTAINER] # Name of cookbook maintainer
-e, [--maintainer-email=MAINTAINER_EMAIL] # Email address of cookbook maintainer
[--no-bundler] # Skips generation of a Gemfile and other Bundler specific support
[--skip-vagrant] # Skips adding a Vagrantfile and adding supporting gems to the Gemfile
[--skip-git] # Skips adding a .gitignore and running git init in the cookbook directory
-c, [--config=PATH] # Path to Berkshelf configuration to use.
-F, [--format=FORMAT] # Output format to use.
# Default: human
-q, [--quiet] # Silence all informational output.
-d, [--debug] # Output debug information
Create a skeleton for a new cookbook](https://image.slidesharecdn.com/theberkshelfway-130509152648-phpapp01/85/The-Berkshelf-Way-22-320.jpg)
![CRACK OPEN
MYFACE::DATABASE_SERVER
node.set[:mysql][:port] = 3309
node.set[:mysql][:tunable][:max_connections] = 1000
include_recipe "myface::_common_system"
include_recipe "mysql::server"](https://image.slidesharecdn.com/theberkshelfway-130509152648-phpapp01/85/The-Berkshelf-Way-45-320.jpg)
![Bringing machine 'default' up with 'virtualbox' provider...
[Berkshelf] Updating Vagrant's berkshelf: '/Users/reset/.berkshelf/vagrant/berkshelf-20130424-40671-t43soz’
Generating chef JSON and uploading...
Running chef-solo...
[2013-04-25T03:01:31+00:00] INFO: *** Chef 10.14.2 ***
[2013-04-25T03:01:32+00:00] INFO: Setting the run_list to ["recipe[minitest-handler::default]", "recipe[myface::default]"] from JSON
[2013-04-25T03:01:32+00:00] INFO: Starting Chef Run for myface-berkshelf
#### CHEF RUN GOES HERE ####
[2013-04-25T03:02:10+00:00] INFO: Chef Run complete in 38.330025825 seconds
[2013-04-25T03:02:10+00:00] INFO: Running report handlers
Run options: -v --seed 41224
# Running tests:
minitesthandler::default#test_0001_runs_no_tests = 0.00 s = .
myface::default#test_0001_runs_no_tests_by_default =
0.00 s = .
Finished tests in 0.002425s, 824.7576 tests/s, 0.0000 assertions/s.
2 tests, 0 assertions, 0 failures, 0 errors, 0 skips
[2013-04-25T03:02:10+00:00] INFO: Report handlers complete
PROVISION NEW ENVIRONMENT IN
MINUTES
$ vagrant up
1 minute 20 secon](https://image.slidesharecdn.com/theberkshelfway-130509152648-phpapp01/85/The-Berkshelf-Way-73-320.jpg)
![Running chef-solo...
[2013-04-25T03:01:31+00:00] INFO: *** Chef 10.14.2 ***
[2013-04-25T03:01:32+00:00] INFO: Setting the run_list to ["recipe[minitest-handler::default]", "recipe[myface::default]"] from JSON
[2013-04-25T03:01:32+00:00] INFO: Starting Chef Run for myface-berkshelf
#### CHEF RUN GOES HERE ####
[2013-04-25T03:02:10+00:00] INFO: Chef Run complete in 0.520326234 seconds
[2013-04-25T03:02:10+00:00] INFO: Running report handlers
Run options: -v --seed 41224
# Running tests:
minitesthandler::default#test_0001_runs_no_tests = 0.00 s = .
myface::default#test_0001_runs_no_tests_by_default =
0.00 s = .
Finished tests in 0.002425s, 824.7576 tests/s, 0.0000 assertions/s.
2 tests, 0 assertions, 0 failures, 0 errors, 0 skips
[2013-04-25T03:02:10+00:00] INFO: Report handlers complete
CONVERGE IN SECONDS
$ vagrant provision
7 seconds](https://image.slidesharecdn.com/theberkshelfway-130509152648-phpapp01/85/The-Berkshelf-Way-75-320.jpg)
The Berkshelf Way
- 1. JAMIE WINSOR| SOFTWARE ENGINEER THE BERKSHELF WAY
- 11. WHAT’S BERKSH ELF? ‣ A COMMAND LINE TOOL ‣ A SOURCE CODE MANAGEMENT TOOL ▾ Similar to ‘Mix’, ‘Go’, or ‘Leiningen’ ‣ A PACKAGE MANAGER ▾ Similar to ‘Gem’, ‘Apt’, or ‘Yum’ ‣ REPLACES PORTIONS OF KNIFE ▾ Cookbook Generator, Cookbook Uploader, Cookbook Downloader
- 12. WHY IS IT CALLED BERKSHELF?
- 14. COOKBOOKS ARE FED INTO CHEF NGINX COOKBOO K 1.2 .3 Chef-Client / Chef-Solo
- 15. RECIPES GO IN COOKBOOKS NGINX COOKBOO K 1.2 .3 RECI PE RECI PE RECI PE
- 16. RECIPES CONTAIN RESOURCE COLLECTIONS DEFAU LT PACKAGE NGINX SERVICE NGINX REPO YUM_REPOSI TORY NGINX YUM_KEY NGINX NGINX COOKBOO K 1.2 .3
- 17. EXAMPLE RESOURCE 1 2 3 package “nginx” do action :install end PACKAGE NGINX
- 19. INSTALL BERKSHELF $ gem install berkshelf Successfully installed berkshelf-1.4.0 1 gem installed
- 20. CREATE A NEW COOKBOOK $ berks cookbook myface create myface/files/default create myface/templates/default create myface/attributes create myface/definitions create myface/libraries create myface/providers create myface/recipes create myface/resources create myface/recipes/default.rb create myface/metadata.rb create myface/LICENSE create myface/README.md create myface/Berksfile create myface/chefignore create myface/.gitignore run git init from "./myface" create myface/Gemfile create myface/Vagrantfile Using myface (0.1.0) at path: '/Users/reset/code/riot-cookbooks/myface'
- 21. CONVERT AN EXISTING COOKBOOK $ berks init create Berksfile create chefignore create .gitignore run git init from "." create Gemfile create Vagrantfile Using old_cookbook (0.1.0) at path: '/Users/reset/code/riot-cookbooks/old_cookbook' Successfully initialized
- 22. STUCK? ASK FOR HELP! $ berks cookbook –h Usage: berks cookbook NAME Options: [--foodcritic] # Creates a Thorfile with Foodcritic support to lint test your cookbook [--chef-minitest] # Creates chef-minitest support files and directories, adds minitest-handler … [--scmversion] # Creates a Thorfile with SCMVersion support to manage versions for continuous integration -L, [--license=LICENSE] # License for cookbook (apachev2, gplv2, gplv3, mit, reserved) # Default: reserved -m, [--maintainer=MAINTAINER] # Name of cookbook maintainer -e, [--maintainer-email=MAINTAINER_EMAIL] # Email address of cookbook maintainer [--no-bundler] # Skips generation of a Gemfile and other Bundler specific support [--skip-vagrant] # Skips adding a Vagrantfile and adding supporting gems to the Gemfile [--skip-git] # Skips adding a .gitignore and running git init in the cookbook directory -c, [--config=PATH] # Path to Berkshelf configuration to use. -F, [--format=FORMAT] # Output format to use. # Default: human -q, [--quiet] # Silence all informational output. -d, [--debug] # Output debug information Create a skeleton for a new cookbook
- 23. A COMPLETE EXAMPLE CHEF- SERVE RCOOKBOO K 1.2 .3 "## Berksfile "## CHANGELOG.md "## CONTRIBUTING.md "## Gemfile "## LICENSE "## README.md "## Thorfile "## Vagrantfile "## attributes $ &## default.rb "## chefignore "## libraries $ "## dev_helper.rb $ &## omnitruck_client.rb "## metadata.rb "## recipes $ "## default.rb $ &## dev.rb &## templates &## default &## chef-server.rb.erb Berksfile metadata.rb
- 25. COOKB OOK METADATA IS IMPORTANT CHEF- SERVE R THE NAME OF THE COOKBOOK
- 26. COOKB OOK CHEF- SERVE R METADATA IS IMPORTANT V2.0.0 THE VERSION OF THE COOKBOOK FOLLOW SEMVER: HTTP://SEMVER.ORG/
- 27. METADATA IS IMPORTANT COOKBO OK YUM V1.2.3 COOKBO OK CHEF- SERVER V1.2.3 COOKBO OK APT V1.2.3 Red Hat CentOS Ubuntu SET YOUR SUPPORTED PLATFORMS CentOS Ubuntu Red Hat
- 28. METADATA IS IMPORTANT COOKBO OK YUM V1.2.3 SET WHAT COOKBOOKS YOU DEPEND ON COOKBO OK CHEF- SERVER V1.2.3 COOKBO OK APT V1.2.3 SUPPORTED PLATFORMS SUPPORTED PLATFORMS SUPPORTED PLATFORMS
- 29. CHEF-SERVER COOKBOOK’S METADATA name "chef-server" maintainer "Opscode, Inc." maintainer_email "cookbooks@opscode.com" license "Apache 2.0" description "Installs and configures Chef Server" long_description IO.read(File.join(File.dirname(__FILE__), 'README.md')) version "2.0.0" depends "apt", ">= 1.7.0" depends "yum", ">= 0.5.0" %w{ ubuntu redhat centos fedora amazon scientific oracle }.each do |os| supports os end
- 30. IT STARTS WITH A BERKSFILE "## Berksfile "## CHANGELOG.md "## CONTRIBUTING.md "## Gemfile "## LICENSE "## README.md "## Thorfile "## Vagrantfile "## attributes $ &## default.rb "## chefignore "## libraries $ "## dev_helper.rb $ &## omnitruck_client.rb "## metadata.rb "## recipes $ "## default.rb $ &## dev.rb &## templates &## default &## chef-server.rb.erb Berksfile site :opscode metadata group :dev do cookbook 'git' end Berksfile
- 31. INSTALLING COOKBOOKS Using chef-server (2.0.0) at path: '/Users/reset/code/riot-cookbooks/chef-server' Installing git (2.5.0) from site: 'http://cookbooks.opscode.com/api/v1/cookbooks' Installing apt (1.9.2) from site: 'http://cookbooks.opscode.com/api/v1/cookbooks' Installing yum (2.2.0) from site: 'http://cookbooks.opscode.com/api/v1/cookbooks' Installing dmg (1.1.0) from site: 'http://cookbooks.opscode.com/api/v1/cookbooks' Installing build-essential (1.4.0) from site: 'http://cookbooks.opscode.com/api/v1/cookbooks' Installing windows (1.8.8) from site: 'http://cookbooks.opscode.com/api/v1/cookbooks' Installing chef_handler (1.1.4) from site: 'http://cookbooks.opscode.com/api/v1/cookbooks' Installing runit (1.1.2) from site: 'http://cookbooks.opscode.com/api/v1/cookbooks' $ berks install
- 32. INSTALLED TO WHERE? $ ls ~/.berkshelf/cookbooks apache2-1.6.0 chef_handler-1.1.4 known_host-0.1.3 nginx-1.3.0 riot_chef-client-2.0.7 tsf4g-0.0.14 apt-1.9.0 couchdb-2.4.0 liquibase-0.0.1 not_system_ruby-0.1.7 riot_coherence-0.10.12 ucspi-tcp-1.0.0 ark-0.0.11 cron-1.0.4 logrotate-0.8.2 ntp-1.1.8 riot_ejabberd-0.0.152 ulimit-0.1.3 artifact-0.10.10 daemontools-1.0.0 logrotate-1.1.0 ohai-1.1.6 riot_ejabberd-0.0.188 users-1.1.4 artifact-0.10.2 database-1.3.12 lvm-0.8.6 openssl-1.0.0 riot_geoip-0.1.0 vim-1.0.2 artifact-1.2.0 database-1.3.4 magic_shell-0.2.0 postgresql-1.0.0 riot_java-0.1.4 windows-1.8.2 artifact-1.3.1 dmg-1.1.0 man-0.7.0 pvpnet_core-2.3.123 riot_java-0.1.8 xfs-1.0.0 artifact-1.4.0 editor-0.2.0 mysql-1.2.4 pvpnet_core-2.3.131 riot_mysql-2.0.1 xml-1.1.2 aws-0.100.0 erlang-1.2.0 mysql-1.3.0 python-1.2.2 riot_rtview-1.0.1 yum-2.1.0 bluepill-1.1.0 gecode-2.0.0 mysql-2.1.0 rbenv-1.3.2 riot_team-0.9.41 zlib-2.0.0 build-essential-1.1.2 git-2.2.0 mysql-2.1.2 riot_activemq-0.9.28 riot_tomcat-1.0.18 build-essential-1.3.4 hornetq-0.0.27 networking_basic-0.0.5 riot_base-2.7.32 runit-0.16.2 chef-client-2.1.4 java-1.5.0 nexus-1.1.0 riot_base-2.7.37 runit-1.0.4 chef-server-1.1.0 java-1.9.0 nginx-1.2.0 riot_base-2.7.49 sudo-1.2.0
- 33. UPLOADING COOKBOOKS Using chef-server (2.0.0) at path: '/Users/reset/code/riot-cookbooks/chef-server' Using git (2.2.0) Using apt (1.9.0) Using yum (2.1.0) Using dmg (1.1.0) Using build-essential (1.3.4) Using windows (1.8.2) Using chef_handler (1.1.4) Using runit (0.16.2) Uploading chef-server (2.0.0) to: 'https://api.opscode.com:443/organizations/vialstudios' Uploading git (2.2.0) to: 'https://api.opscode.com:443/organizations/vialstudios' Uploading apt (1.9.0) to: 'https://api.opscode.com:443/organizations/vialstudios' Uploading yum (2.1.0) to: 'https://api.opscode.com:443/organizations/vialstudios' Uploading dmg (1.1.0) to: 'https://api.opscode.com:443/organizations/vialstudios' Uploading build-essential (1.3.4) to: 'https://api.opscode.com:443/organizations/vialstudios' Uploading windows (1.8.2) to: 'https://api.opscode.com:443/organizations/vialstudios' Uploading chef_handler (1.1.4) to: 'https://api.opscode.com:443/organizations/vialstudios' Uploading runit (0.16.2) to: 'https://api.opscode.com:443/organizations/vialstudios' $ berks upload
- 38. SERVICES ARE MADE UP OF COMPONENTS 1 app servers 2 cache servers 3 database servers 4 background workers 5 load balancers 6 etc
- 39. APPLY A DESIGN PATTERN
- 42. COMPONENTS OF MYFACE 1 load balancer 2 application server proxy 3 caching service 4 application server 5 background worker 6 database server
- 45. CRACK OPEN MYFACE::DATABASE_SERVER node.set[:mysql][:port] = 3309 node.set[:mysql][:tunable][:max_connections] = 1000 include_recipe "myface::_common_system" include_recipe "mysql::server"
- 46. PUBLIC/PRIVATE RECIPES PUBLIC RECIPE ‣ What you put in the run_list of a node ‣ Documented in the README ‣ Documented in the metadata PRIVATE RECIPE ‣ Not exposed to end user ‣ Should never be put in the run_list of a node ‣ Always included into other recipes ‣ Documented in the code for other developers
- 47. WHAT’S IN MYFACE::_COMMON_SYSTEM? include_recipe "ntp::default" # . . . MAYBE SETUP SOME USERS . . . # . . . MAYBE SETUP SOME ADDITIONAL REPOS . . .
- 48. WORK IN VERTICALS FROM THE OUTSIDE-INFAVOR DATA- DRIVEN COOKBOOKS THE BERKSHE LF WAY
- 50. TUNABLES OF MYFACE 1 listen on a different port or bind address 2 set memory usage of workers 3 any app config 4 set services to started or stopped states
- 51. 3 WAYS TO CONFIGURE ATTRIBUTES DATA BAGS ENCRYPTED DATA BAGS
- 52. FAVOR CONFIGURING WITH ATTRIBUTES ‣ Path of least resistance for cookbook consumers ‣ Provide sensible defaults in the Attribute files ▾ Consider creating an attribute file for each recipe ‣ Can configure your application on an environment level
- 53. WHEN TO USE DATA BAGS USERS GROUPS YUM/APT THINGS CONFIGURED BY "BASE" COOKBOOK (IF YOU HAVE ONE) ORGANIZATION LEVEL CONFIGURATIONS
- 54. USE ENCRYPTED DATA BAGS WHEN STORING SENSITIVE DATA
- 55. WORK IN VERTICALS FROM THE OUTSIDE-INFAVOR DATA- DRIVEN COOKBOOKS WRITE WELL- ENCAPSULATED COOKBOOKS THE BERKSHE LF WAY
- 58. NOT VERSIONED CAN’T BE PACKAGED / DISTRIBUTED NOT NAMESPACED STAHP USING ROLES THEY ARE ORGANIZATION LEVEL DATA ADDITIONAL SETUP COMPLEXITY FOR THE USER
- 60. USE DATA BAGS WHERE THEY MAKE SENSE DO YOU NEED TO STORE ORGANIZATION-WIDE DATA? DO YOU NEED TO STORE SENSITIVE DATA? Use one Use an encrypted one
- 61. USE DATA BAGS WHERE THEY MAKE SENSE MOST IMPORTANTLY… VALIDATE YOUR DATA BAGS!
- 62. 5 include_recipe "myface::app_proxy" 6 include_recipe "myface::app_server" 7 include_recipe "myface::cache_server" 8 include_recipe "myface::worker_pool" 9 include_recipe "myface::database" DEFAULTS RECIPE IS SPECIAL
- 63. LIGHTWEIGHT RESOURCES AND PROVIDERS (LWRP) LIBRARIES DEFINITIONS COOKBOOKS ARE MORE THAN RECIPES COOKBOOKS DON’T NEED TO HAVE RECIPES
- 65. LIBRARY COOKBOOKS ‣ Might have recipes, might not ‣ Contain LWRPs, Definitions, or Libraries useful to other cookbook authors ‣ YUM Cookbook is a perfect example ▾ Contains a yum_key LWRP for adding repository keys to yum ▾ Contains a yum_repository LWRP for adding repositories to yum ▾ Contains some recipes to configure commonly used public yum repositories on your machine
- 67. WRAPPER COOKBOOKS ‣ Super lightweight ‣ Contain recipes ‣ Contain attribute overrides ‣ Riot-Java is a perfect example ▾ Contains a Oracle Java 6 recipe ▾ Contains a Oracle Java 7 recipe ▾ An abstraction on top of the Java cookbook for Riot engineers ▾ Overrides the default “install flavor” for Java ▾ Overrides the default location of the Java artifacts to an internal location ▾ NOT A FORK OF THE JAVA COOKBOOK
- 68. WORK IN VERTICALS FROM THE OUTSIDE-INFAVOR DATA- DRIVEN COOKBOOKS WRITE WELL- ENCAPSULATED COOKBOOKS HAVE SHORT ITERATION LOOPS THE BERKSHE LF WAY
- 69. OLD WORKFLOW EDIT COOKBOOK UPLOAD COOKBOOK PROVISION MACHINE BOOTSTRAP MACHINE RUN CHEF CLIENT SSH AND VALIDATE 5-30 minutes
- 70. NEW WORKFLOW EDIT COOKBOOK VAGRANT UP / VAGRANT PROVISION TEST / SSH AND VALIDATE 5 seconds – 1 minute
- 72. BERKSHELF VAGRANT PLUG-IN $ vagrant plugin install vagrant-berkshelf Installing the 'vagrant-berkshelf' plugin. This can take a few minutes... Installed the plugin 'vagrant-berkshelf (1.2.0)'!
- 73. Bringing machine 'default' up with 'virtualbox' provider... [Berkshelf] Updating Vagrant's berkshelf: '/Users/reset/.berkshelf/vagrant/berkshelf-20130424-40671-t43soz’ Generating chef JSON and uploading... Running chef-solo... [2013-04-25T03:01:31+00:00] INFO: *** Chef 10.14.2 *** [2013-04-25T03:01:32+00:00] INFO: Setting the run_list to ["recipe[minitest-handler::default]", "recipe[myface::default]"] from JSON [2013-04-25T03:01:32+00:00] INFO: Starting Chef Run for myface-berkshelf #### CHEF RUN GOES HERE #### [2013-04-25T03:02:10+00:00] INFO: Chef Run complete in 38.330025825 seconds [2013-04-25T03:02:10+00:00] INFO: Running report handlers Run options: -v --seed 41224 # Running tests: minitesthandler::default#test_0001_runs_no_tests = 0.00 s = . myface::default#test_0001_runs_no_tests_by_default = 0.00 s = . Finished tests in 0.002425s, 824.7576 tests/s, 0.0000 assertions/s. 2 tests, 0 assertions, 0 failures, 0 errors, 0 skips [2013-04-25T03:02:10+00:00] INFO: Report handlers complete PROVISION NEW ENVIRONMENT IN MINUTES $ vagrant up 1 minute 20 secon
- 75. Running chef-solo... [2013-04-25T03:01:31+00:00] INFO: *** Chef 10.14.2 *** [2013-04-25T03:01:32+00:00] INFO: Setting the run_list to ["recipe[minitest-handler::default]", "recipe[myface::default]"] from JSON [2013-04-25T03:01:32+00:00] INFO: Starting Chef Run for myface-berkshelf #### CHEF RUN GOES HERE #### [2013-04-25T03:02:10+00:00] INFO: Chef Run complete in 0.520326234 seconds [2013-04-25T03:02:10+00:00] INFO: Running report handlers Run options: -v --seed 41224 # Running tests: minitesthandler::default#test_0001_runs_no_tests = 0.00 s = . myface::default#test_0001_runs_no_tests_by_default = 0.00 s = . Finished tests in 0.002425s, 824.7576 tests/s, 0.0000 assertions/s. 2 tests, 0 assertions, 0 failures, 0 errors, 0 skips [2013-04-25T03:02:10+00:00] INFO: Report handlers complete CONVERGE IN SECONDS $ vagrant provision 7 seconds
- 76. WORK IN VERTICALS FROM THE OUTSIDEFAVOR DATA- DRIVEN COOKBOOKS WRITE WELL- ENCAPSULATED COOKBOOKS HAVE SHORT ITERATION LOOPS THE BERKSHE LF WAY
- 78. CONTRIBUTE GITHUB.COM/ RIOTGAMES/ BERKSHELF HOW CAN YOU HELP? OR JOIN US WWW.RIOTGAMES.COM/CAREERS