![24 Confidential
Dealing with outliers
§ Accept work around
ssh.exec( "
esxcfg-vmknic -i #{opts[:new_mgmt_ip]}
-n #{opts[:new_mgmt_mask]}
-p 'Management Network' &&
esxcfg-route -a default #{opts[:new_mgmt_gw]} &&
esxcfg-vswitch -p 'Management Network'
-v #{opts[:new_mgmt_vlan]} vSwitch0
" )
§ Last resort, file a ticket
t = ServiceNow::Request.new(
:subject => ”Please click yes"
)
ServiceNow.create(t)](https://image.slidesharecdn.com/puppetconf2013vchsandpuppet-130824115716-phpapp01/85/PuppetConf-2013-vCloud-Hybrid-Service-and-Puppet-24-320.jpg)
![29 Confidential
Where’s my HOME?
§ Exec specify HOME.
§ Providers Fix:
if respond_to? :has_command
has_command(:brew, "/usr/local/bin/brew") do
environment({ 'HOME' => ENV['HOME'] })
end
else
commands :brew => "/usr/local/bin/brew”
end](https://image.slidesharecdn.com/puppetconf2013vchsandpuppet-130824115716-phpapp01/85/PuppetConf-2013-vCloud-Hybrid-Service-and-Puppet-29-320.jpg)
PuppetConf 2013 vCloud Hybrid Service and Puppet
- 1. © 2012VMware Inc. All rights reserved Confidential VMware vCloud Hybrid Service and Puppet Nan Liu, Sr. Systems Engineer - VMware Inc
- 2. 2 Confidential Overview § VMware vCloud Hybrid Service § Automation with Puppet § Lesson’s learned the hard way
- 3. 3 Confidential vCloud Hybrid Service VMware vCloud Hybrid Service Your Data Center Software-Defined Data Center VMware vSphere & vCloud Suite Existing & New Apps Seamless Networking Common management One Support call
- 4. 4 Confidential vCloud Hybrid Service (vCHS) § Customer:
- 5. 5 Confidential vCloud Hybrid Services § Engineer: -------------- 1 --------------- Init VPC - Engine - API Call ------------- 3 -------------- Install ESXi Engine Broker Plugin - Razor Inputs: - Razor IP - Blade - Hostname - Network Information Tasks: - Create Policy - Boot Blade ------------- 3 -------------- Prep TLM Engine Broker Plugin - Puppet vCenter Inputs: - TLM VCSA IP - TLM VCSA Creds - TLM DVS - Portgroup Name - Portgroup VLAN - Portgroup Settings - Ports: 128 - Load Balancing: IP Hash - Reset at Disconnect: Enabled Tasks: - Create PGs - d#p#v#-dmz-pg-## - d#p#v#-esx-pg-## - d#p#v#-pvt-pg-## ----------- 4 ------------ Deploy VPC VCSA Engine Broker Plugin - OVFTool Inputs: - OVA - TLM vCenter IP - TLM vCenter Creds - Cluster - Datastore - VM Name - Network - VM Hardware Settings - Memory: 16384 Tasks: - Deploy VPC VCSA - d#p#v#-mgmt-vc0 - Update VM Hardware ----------- 4 ------------ Deploy VPC VSE Pub Pair Engine Broker Plugin - Puppet vShield Inputs: - TLM VSM IP - TLM VSM Creds - Edge Settings - Name - Hostname - Enable HA - Declare Dead Time: 6 - Interface 1 - Heartbeat IP Addresses - TLM Datacenter - TLM Cluster - TLM Shared Datastore - Size: Compact - Interface Settings - Interface 0 - Name - Type - Portgroup - Subnet Settings - Interface 1 - Name - Type - Portgroup - Subnet Settings Tasks: - Create VPC Pub Edge - d#p#v#-mgmt-vse-pub ----------- 4 ------------ Deploy VPC VSE Priv Pair Engine Broker Plugin - Puppet VMware-vShield Inputs: - TLM VSM IP - TLM VSM Creds - Edge Settings - Name - Hostname - Enable HA - Declare Dead Time: 6 - Interface 2 - Heartbeat IP Addresses - TLM Datacenter - TLM Cluster - TLM Shared Datastore - Size: Compact - Interface Settings - Interface 0 - Name: InterVPC - Type - Portgroup - Subnet Settings - Interface 1 - Name: DMZ - Type - Portgroup - Subnet Settings - Interface 2 - Name: PVT - Type - Portgroup - Subnet Settings - Interface 3 - Name: ESX - Type - Portgroup - Subnet Settings Tasks: - Create VPC Priv Edge - d#p#v#-mgmt-vse-priv ----------- 5 ------------ Configure VPC VSE Pub Pair Engine Broker Plugin - Puppet vShield Inputs: - TLM VSM IP - TLM VSM Creds - Edge Settings - DNS Settings - Firewall Settings - IPsets - Application Groups - Applications - Firewall Rules - Load Balancer Settings - Enable - Pools - VIPs - Default Route - Syslog Tasks: - Configure VPC VSE Pub Pair ----------- 5 ------------ Configure VPC VSE Priv Pair Engine Broker Plugin - Puppet VMware-vShield Inputs: - TLM VSM IP - TLM VSM Creds - Edge Settings: - Syslog - DNS - Firewall Settings - IPsets - Application Groups - Applications - Firewall Rules - Load Balancer Settings - Enable - Pools - VIPs - Default Route Tasks: - Configure VPC VSE Priv Pair ----------- 7 ------------ Init VPC VCSA Engine Broker Plugin - Puppet VMware-VCSA Inputs: - Credentials - NTP Settings - Syslog Settings - DB Settings - Sizing: Large Tasks: - Init VPC VCSA ----------- 4 ------------ Deploy VPC VSM Engine Broker Plugin - OVFTool Inputs: - OVA - TLM vCenter IP - TLM vCenter Creds - Cluster - Datastore - VM Name - Network Tasks: - Deploy VPC VSM - d#p#v#-mgmt-vsm0 ----------- 9 ------------ Configure VPC VSM Engine Broker Plugin - Puppet VMware-vShield Inputs: - VPC VSM IP - VPC VSM Creds - VPC VCSA IP - VPC VCSA Creds - NTP - Syslog Tasks: - Configure VPC VSM ----------- 4 ------------ Deploy VPC vCloud Engine Broker Plugin - ?? Inputs: - TLM vCenter IP - TLM vCenter Creds - Cluster - Datastore - VM Names - VM Settings - Network Tasks: - Deploy VPC vCloud Cells - Deploy VPC vCloud NFS - Deploy VPC vCloud DB -------------- 2 ----------------- Reserve VPC Engine Broker Plugin: - Rez Inputs: - VPC # ----------- 8 ------------ Configure VPC vCenter Engine Broker Plugin - Puppet VMware-vCenter Inputs: - VPC VCSA IP Address - VPC VCSA Credentials - Licenses - vCenter - ESXi - vCloud Net & Sec - Retention Policies - Task: Enabled - Event: Enabled - Datacenter Name - Cluster Settings - Name - DRS Settings - EVC Settings? - DVS Configuration - Name: d#p#v# - Settings - Uplinks: 4 - MTU: 9000 - Enable NIOC - Portgroup settings - Name - VLAN - Settings -Tasks: - Configure VPC VCSA - Configure DVS ----------- 7 ------------ Init VPC VSM Engine Broker Plugin - ?? Inputs: - VPC VSM Network Settings Tasks: - Init VPC VSM ----------- 6 ------------ VPC VCSA Port Check Engine Broker Plugin - Util Inputs: - VPC VCSA IP Tasks: - VPC VCSA Port Check ----------- 6 ------------ VPC VSM Port Check Engine Broker Plugin - Util Inputs: - VPC VSM IP Tasks: - VPC VSM Port Check ----------- 6 ------------ Verify VPC ESXi Engine Broker Plugin - Puppet Util? Inputs: - VPC ESXi IP - VPC ESXii Creds Tasks: - Verify VPC ESXi ----------- 9 ------------ Add ESXi to VPC Datacenter Engine Broker Plugin - Puppet VMware-vCenter Inputs: - VPC ESXi IP - VPC ESXii Creds - VPC VCSA IP - VPC VCSA Creds - VPC Datacenter Name Tasks: - Add ESXi to VPC Datacenter ----------- 10 ------------ Configure VPC ESXi Engine Broker Plugin - Puppet VMware-vCenter Inputs: - VPC VCSA IP - VPC VCSA Creds - DNS Settings - NTP Settings - Syslog Settings - VMK Configuration - vMotion - Storage - FT - ??? Tasks: - Configure ESXi - Add to DVS - Add VMK Networking ----------- 11 ------------ Add Storage to VPC ESXi Engine Broker Plugin - Puppet VMware-vCenter Inputs: - VPC VCSA IP - VPC VCSA Creds - iSCSI Settings - LUN Information Tasks: - Configure iSCSI - Add LUNs ------------- 3 --------------- Configure VNX VPC Storage Group Engine Broker Plugin: - Storage Controller Inputs: - VPC # - LUN IDs - ESXi iSCSI information Tasks: - Create VPC Storage Group - Configure VNX Host registrations - Add LUNs to VPC Storage Group ----------- 6 ------------ VPC vCloud Cell Port Check Engine Broker Plugin - Util Inputs: - VPC vCloud Cell IP Tasks: - VPC vCloud Cell Port Check ----------- 6 ------------ VPC vCloud Cell Port Check Engine Broker Plugin - Util Inputs: - VPC vCloud Cell IP Tasks: - VPC vCloud Cell Port Check ----------- 6 ------------ VPC vCloud NFS Port Check Engine Broker Plugin - Util Inputs: - VPC vCloud Cell IP Tasks: - VPC vCloud NFS Port Check ----------- 6 ------------ VPC vCloud DB Port Check Engine Broker Plugin - Util Inputs: - VPC vCloud Cell IP Tasks: - VPC vCloud DB Port Check ----------- 7 ------------ Configure VPC vCloud NFS Engine Broker Plugin - ?? Inputs: - VPC vCloud NFS IP - VPC vCloud NFS Creds - NFS Export Settings - ??? Tasks: - Config VPC vCloud NFS ----------- 7 ------------ Configure VPC vCloud DB Engine Broker Plugin - ?? Inputs: - VPC vCloud DB IP - VPC vCloud DB Creds - vCloud Database Config - ??? Tasks: - Configure VPC vCloud DB ----------- 8 ------------ Configure VPC vCloud Cell Engine Broker Plugin - ?? Inputs: - VPC vCloud Cell IP - VPC vCloud Cell Creds - VPC vCloud NFS Config - VPC vCloud Installation Responses.properties - VPC vCloud Cell Cert - NTP Settings Tasks: - Install and Configure NTP - Install VPC vCloud Cert - Configure vCD - Configure vCD Transfer Service ----------- 9 ------------ Configure VPC vCloud Cell Engine Broker Plugin - ?? Inputs: - VPC vCloud Cell IP - VPC vCloud Cell Creds - VPC vCloud Installation Responses.properties - VPC vCloud NFS Config - VPC vCloud Cell Cert - NTP Settings Tasks: - Install and Configure NTP - Install VPC vCloud Cert - Configure vCD - Configure vCD Transfer Service ----------- 12 ------------ Create VPC VM Storage Profile Engine Broker Plugin - Puppet VMware-vCenter Inputs: - VPC VCSA IP - VPC VCSA Creds - Storage Profile Name Tasks: - Create VPC VM Storage Profile - Tag VPC Datastores ----------- 13 ------------ Add VPC vCenter to vCloud Engine Broker Plugin - Puppet VMware-vCloud Inputs: - VPC vCloud IP - VPC vCloud Creds - VPC VCSA IP - VPC VCSA Creds - ??? Tasks: - Add VPC vCenter to VPC vCloud ----------- 10 ------------ Configure VXLAN Engine Broker Plugin - Puppet VMware-vShield Inputs: - VPC VSM IP - VPC VSM Creds - Multicast Information - VPC DVS Information - VPC Cluster - ??? Tasks: - Configure VXLAN ----------- 11 ------------ Reconfigure VXLAN VMK Ports Engine Broker Plugin - Puppet VMware-vCenter Inputs: - VPC VCSA IP - VPC VCSA Creds - VXLAN VMK Network Settings Tasks: - Configure VXLAN VMK Ports ----------- 14 ------------ Configure VPC vCloud Engine Broker Plugin - Puppet VMware-vCloud Inputs: - VPC vCloud IP - VPC vCloud Creds - Provider VDC Name - External Org Network Name - ??? Tasks: - Configure VPC vCloud ----------- 12 ------------ Configure VPC Cluster HA Engine Broker Plugin - Puppet VMware-vCenter Inputs: - VPC VCSA IP - VPC VCSA Creds - VPC ESXi Hosts - VPC Cluster - HA Settings - Failure: percentage Tasks: - Add VPC ESXi hosts to Cluster - Configure VPC Cluster HA Settings ----------- 5 ------------ Prep VPC vCloud Cell Engine Broker Plugin - Puppet vCenter - ?? Inputs: - VPC VCSA IP - VPC VCSA Cred - VPC vCloud Cell Name - Static Routes Tasks: - PowerOn VM - Set Hostname - Set Static Routes - Configure Networking ----------- 5 ------------ Prep VPC VCSA Engine Broker Plugin - ?? Inputs: - Credentials - Network Settings Tasks: - Configure Networking ----------- 5 ------------ Prep VPC vCloud Cell Engine Broker Plugin - Puppet vCenter - ?? Inputs: - VPC VCSA IP - VPC VCSA Cred - VPC vCloud Cell Name - Static Routes Tasks: - PowerOn VM - Set Hostname - Set Static Routes - Configure Networking ----------- 5 ------------ Prep VPC VSM Engine Broker Plugin - ?? Inputs: - VPC VCSA IP - VPC VCSA Cred - VPC VSM name - Network Settings Tasks: - PowerOn VM - Configure Networking
- 6. 6 Confidential vCloud Hybrid Service § Start your engine:
- 7. 7 Confidential Click Fail Add VPC Hosts to VCSA § Continuing from the vCenter view of the vSphere Web Client § In the main content pane, click Related Objects > Clusters § Click on VPC on the list § Click the second plus icon to add a host • Host name: Enter first host FQDN • Location: Confirm the location is set. • Click NEXT § … § Repeat Manually?
- 8. 8 Confidential Problem Challenges: § Reduce deployment time § Reduce complexity § Scale … Fast Solution: 1. Automate 2. See first rule
- 10. 10 Confidential Infrastructure Services Server & OS Services (Puppet) § NTP § Syslog § RabbitMQ § …
- 11. 11 Confidential Automation Challenges § Service APIs: • vCenter: vSphere API (soap) • vCNS: vShield API (REST) • vCD: vCD API (REST) § vCenter/vCNS servers are appliances
- 12. 12 Confidential Why not ‘puppet device’? § Limited to one device at a time: Puppet Management Server VMware vCenter VMware vShield Datacenter ESX ESX ESX Datacenter ESX ESX vSphere API vShield API Edge
- 13. 13 Confidential Solution: Take the road less traveled § Everything is a Native Resource § Transport to the rescue • vCenter/vCNS Appliance: SSH • vCenter API: RbVmomi • vShield API: rest-client + subset of savon project • vCD API: rest-client
- 14. 14 Confidential Transport Resource § Credentials § Connectivity Options § Multiple connections
- 16. 16 Confidential Transport Puppet Management Server VMware vCenter SSH Puppet Management Server VMware vCenter vSphere API Datacenter Folders Datacenter ESX
- 17. 17 Confidential vCenter Appliance Resource
- 20. 20 Confidential Transport § Persistent shared connection § Connection cleanup after catalog apply § Open to supporting additional transport • VMware-RabbitMQ (REST)
- 21. 21 Confidential Modules Puppet Enterprise Users (PE 2.7): § http://forge.puppetlabs.com/vmware Puppet Developers (Developing Puppet 3): § http://github.com/vmware/vmware-vmware_lib § http://github.com/vmware/vmware-vcsa § http://github.com/vmware/vmware-vcenter § http://github.com/vmware/vmware-vshield
- 22. 22 Confidential Lessons Learned § Working with APIs § Puppet 2.7.x -> Puppet 3.x § Puppet Wat?
- 23. 23 Confidential Working with APIs § Functionalities not always in API* § Dealing with API versions § Metaprogramming * Even if you are @lamw :)
- 24. 24 Confidential Dealing with outliers § Accept work around ssh.exec( " esxcfg-vmknic -i #{opts[:new_mgmt_ip]} -n #{opts[:new_mgmt_mask]} -p 'Management Network' && esxcfg-route -a default #{opts[:new_mgmt_gw]} && esxcfg-vswitch -p 'Management Network' -v #{opts[:new_mgmt_vlan]} vSwitch0 " ) § Last resort, file a ticket t = ServiceNow::Request.new( :subject => ”Please click yes" ) ServiceNow.create(t)
- 25. 25 Confidential Testing API versions § puppet apply --libdir=/dev/null § bundler exec + Gemfile source 'file:///opt/repo/’ source 'https://rubygems.org’ gem 'facter’ gem 'puppet', '3.2.2’ gem 'gyoku', '1.0.0z2’ gem 'hashdiff’ gem 'net-ssh’ gem 'nokogiri’ gem 'nori', '1.1.4’ gem 'rbvmomi', '1.6.0.z1’ gem 'rest-client’ gem 'pry'
- 26. 26 Confidential Metaprogramming Good § Reduces boiler plate code § No more API ‘transfer’ bugs § API reference = Resource reference Bad § Difficult to debug § Fixing API issues results in surprises § Not all APIs are designed to be idempotent • action => { :create, :modify }
- 27. 27 Confidential Puppet 3 Upgrade § What the scope? § Where’s my HOME? § Ruby 1.9.3
- 28. 28 Confidential What the scope? § Fully qualify ::class::var § Fix your template @var, scope.lookup(‘::class::var’)
- 29. 29 Confidential Where’s my HOME? § Exec specify HOME. § Providers Fix: if respond_to? :has_command has_command(:brew, "/usr/local/bin/brew") do environment({ 'HOME' => ENV['HOME'] }) end else commands :brew => "/usr/local/bin/brew” end
- 30. 30 Confidential Puppet Wat Boolean: § adrientthebo/boolean § vmware_lib property § Symbool in Hash What the undef? § Careful about behavior ENC Data § ENC integer .to_s
- 31. 31 Confidential Where we are Today § Deploy VPC: # § 95% reduction in deployment time § Configuration Management = Version § Metrics: • 47 Modules • 70 Custom Resources • 1400 Resources
- 32. 32 Confidential Work at VMware Challenges: § Software Defined Datacenter • Data driven configuration management § Software Defined Networking • vCNS, Nicira VSX § Scale + Speed • 10X, 100X, 1000X ? • Faster !!! Now !!!
- 33. 33 Confidential Thanks! § Nicholas Weaver § Randy Brown § Shawn Holland § Floyd Arguello § David Scherer § Ryan Zenker § Justin Guidroz § Dan Pittman § Branan Purvine-Riley § Zach Leslie § vCHS R&D Team
- 34. 34 Confidential Q & A