Paul Angus (ShapeBlue) - Push infrastructure with Ansible #DOXLON
2 likes2,190 views
The document outlines how Shapeblue uses Ansible to automate the deployment of cloud infrastructures, specifically with Apache CloudStack, highlighting its capabilities, advantages, and practical implementations. It emphasizes Ansible's agentless nature and ease of use, enabling quick setup and management of cloud environments at scale. Various workflows and coding examples illustrate the process of utilizing Ansible for cloud environment orchestration, testing, and deployment.
![@CloudyAngus
It’s
OK
it
has
an
API
cloudmonkey create network name=DemoNet displaytext=DemoNet zoneid=$ZONEID
networkofferingid=$NEWORKOFFERINGID account=$ACCOUNTNAME domainid=$DOMAINID
cloudmonkey list networks domainid="$DOMAINID" filter=id,name | jq '.network[] |
select(.name=="DemoNet") | .id‘
cloudmonkey deploy virtualmachine networkids="$NETWORKID"
serviceofferingid="$SERVICEOFFERINGID" templateid="$TEMPLATEID" zoneid="$ZONEID"
account="$ACCOUNTNAME" domainid="$DOMAINID" startvm=true displayname=HTTPServer1
displayvm=true name=HTTPServer1
What
is
CloudStack](https://image.slidesharecdn.com/ansible-doxlon-150303092550-conversion-gate01/85/Paul-Angus-ShapeBlue-Push-infrastructure-with-Ansible-DOXLON-10-320.jpg)
![@CloudyAngus
#
Copyright
(C)
ShapeBlue
Ltd
-‐
All
Rights
Reserved
#
Unauthorized
copying
of
this
file,
via
any
medium
is
strictly
prohibited
#
Proprietary
and
confidential
#
Released
by
ShapeBlue
<info@shapeblue.com>,
April
2014
[cloudstack]
name=cloudstack
baseurl=http://{{
acs_build_repo
}}/{{
acs_build_path
}}
enabled=1
gpgcheck=1
cloudstack.repo.j2](https://image.slidesharecdn.com/ansible-doxlon-150303092550-conversion-gate01/85/Paul-Angus-ShapeBlue-Push-infrastructure-with-Ansible-DOXLON-23-320.jpg)
![@CloudyAngus
[management_hosts:children]
cloudstack_manager_hosts
loadbalancer_hosts
deployment_hosts
db_hosts
[kvm_hosts:children]
pod1_cluster1
pod1_cluster2
[pod1_cluster1]
10.101.1.13 pxemac=00:1c:c0:ec:b5:bc hostname=kvm-‐p1-‐c1-‐1 mgmt_ip=10.101.1.13
mgmt_netmask=255.255.255.0
Host
File](https://image.slidesharecdn.com/ansible-doxlon-150303092550-conversion-gate01/85/Paul-Angus-ShapeBlue-Push-infrastructure-with-Ansible-DOXLON-30-320.jpg)
![@CloudyAngus
Host
File
[mysql_master_host]
10.101.1.6 pxemac=00:50:56:8E:19:20 hostname=mysqlmasterhost mgmt_ip=10.101.1.6
mgmt_netmask=255.255.255.0 serverid=1
[pod1_cluster1]
10.101.1.13 pxemac=00:1c:c0:ec:b5:bc hostname=kvm-‐p1-‐c1-‐1 mgmt_ip=10.101.1.13
10.101.1.14 pxemac=00:1c:c0:ec:b5:43 hostname=kvm-‐p1-‐c1-‐2 mgmt_ip=10.101.1.14
Creating
PXE
files](https://image.slidesharecdn.com/ansible-doxlon-150303092550-conversion-gate01/85/Paul-Angus-ShapeBlue-Push-infrastructure-with-Ansible-DOXLON-31-320.jpg)
![@CloudyAngus
File
format:
AA:BB:CC:DD:EE:FF
must
be
01-‐aa-‐bb-‐cc-‐dd-‐ee-‐ff
-‐ name: create centos pxeconfig files
template: src=pxelinuxcfg-‐centos6.j2 dest=/ptpboot/pxelinux.cfg/01-‐{{ hostvars[item]['pxemac'] | lower |
replace(":","-‐") }}
with_items:
-‐ "{{ groups.management_hosts }}“
-‐ name: create KVM pxeconfig files
template: src=pxelinuxcfg-‐kvm.j2 dest=/ptpboot/pxelinux.cfg/01-‐{{ hostvars[item]['pxemac'] | lower |
replace(":","-‐") }}
with_items:
-‐ "{{ groups.kvm_hosts }}"
Creating
PXE
files](https://image.slidesharecdn.com/ansible-doxlon-150303092550-conversion-gate01/85/Paul-Angus-ShapeBlue-Push-infrastructure-with-Ansible-DOXLON-32-320.jpg)
![@CloudyAngus
# Ansible built pxeconfig script
DEFAULT menu.c32
menu ctle ShapeBlue CSForge Deployment
cmeout 10
LABEL {{ centos_iso_version }}-‐KVM
kernel htp://{{ hostvars[inventory_hostname]['mgmt_ip'] }}/{{ centos_iso_version }}/
isolinux/vmlinuz
append initrd=htp://{{ hostvars[inventory_hostname]['mgmt_ip'] }}/
{{ centos_iso_version }}/isolinux/initrd.img ramdisk_size=15491 ip=dhcp ksdevice={{ ks_device }}
ks=htp://{{ hostvars[inventory_hostname]['mgmt_ip'] }}/{{ pxe_files_dir }}/{{ kvm_ks_file }}
MENU LABEL {{ centos_iso_version }} for KVM
pxelinuxcfg-‐kvm.j2](https://image.slidesharecdn.com/ansible-doxlon-150303092550-conversion-gate01/85/Paul-Angus-ShapeBlue-Push-infrastructure-with-Ansible-DOXLON-33-320.jpg)
![@CloudyAngus
# Specifies the keyboard layout
keyboard {{ keyboard_lang }}
# Used with an HTTP install to specify where the install files are located
url -‐-‐url htp://{{ hostvars[inventory_hostname]['mgmt_ip'] }}/{{ centos_iso_version }}
# Assign a stacc IP address upon first boot & set the hostname
network -‐-‐onboot yes -‐-‐device {{ ks_device }} -‐-‐bootproto dhcp -‐-‐noipv6
# Set the root password
rootpw {{ mgmt_root_password }}
mkdir /root/.ssh
curl htp://{{ hostvars[inventory_hostname]['mgmt_ip'] }}/{{ publickey_file_name }} >> /root/.ssh/authorized_keys
kickstart.j2](https://image.slidesharecdn.com/ansible-doxlon-150303092550-conversion-gate01/85/Paul-Angus-ShapeBlue-Push-infrastructure-with-Ansible-DOXLON-34-320.jpg)
![@CloudyAngus
opcon subnet-‐mask {{ dhcp_netmask }};
opcon routers {{ dhcp_router }};
opcon broadcast-‐address {{ dhcp_broadcast }};
opcon domain-‐name-‐servers {{ int_dns_servers | join(", ") }};
opcon domain-‐name "{{ dns_domain }}";
subnet {{ dhcp_subnet }} netmask {{ dhcp_netmask }} {
range {{ dhcp_range_starcp }} {{ dhcp_range_endip }};
# KVM Hosts
{% for host in groups['kvm_hosts'] %}
host {{ hostvars[host]['hostname'] }} {
hardware ethernet {{ hostvars[host]['pxemac'] | lower }};
fixed-‐address {{ hostvars[host]['mgmt_ip'] }};
}
{% endfor %}
Creating
dhcpd.conf](https://image.slidesharecdn.com/ansible-doxlon-150303092550-conversion-gate01/85/Paul-Angus-ShapeBlue-Push-infrastructure-with-Ansible-DOXLON-35-320.jpg)
![@CloudyAngus
v Reverse Zone – Format:
2 IN PTR blah1.domain.com.
; Address Records
{% for host in groups['management_hosts'] %}
{{ hostvars[host]['mgmt_ip']|split('.')[3] }} IN PTR {{ hostvars[host]['hostname'] }}.
{% endfor %}
{% for host in groups['xenserver_hosts'] %}
{{ hostvars[host]['mgmt_ip']|split('.')[3] }} IN PTR {{ hostvars[host]['hostname'] }}.
{% endfor %}
DNS
Entries
into
Zone
File
–
Jinja2](https://image.slidesharecdn.com/ansible-doxlon-150303092550-conversion-gate01/85/Paul-Angus-ShapeBlue-Push-infrastructure-with-Ansible-DOXLON-36-320.jpg)
Paul Angus (ShapeBlue) - Push infrastructure with Ansible #DOXLON
- 1. Push Infrastructure with Ansible Paul Angus Cloud Architect & Chief Technology Strategist ShapeBlue paul.angus@shapeblue.com Twitter: @CloudyAngus
- 2. @CloudyAngus v How we use Ansible to push out entire cloud infrastructures v A bit of background v How we use Ansible v Work flow v Code snippets Push Infrastructure with Ansible
- 3. @CloudyAngus v Cloud Architect & Chief Technology Strategist for ShapeBlue v Apache CloudStack Committer v Specialise in…. v Designing and deploying enterprise and public clouds v Helping organisations use their cloud v Involved with CloudStack before donation to Apache Foundation v Designed clouds for Orange, TomTom, PaddyPower, Ascenty, BSkyB About Me Unofficial Ansible Evangelist/Cheerleader
- 4. @CloudyAngus “ShapeBlue are expert builders of public & private clouds. They are the leading global Apache CloudStack integrator & consultancy” About ShapeBlue
- 6. @CloudyAngus Push Infrastructure with Ansible What is CloudStack?
- 7. @CloudyAngus CloudStack is a open source IaaS platform. Hypervisor agnostic -‐ KVM, vSphere, XenServer, LXC, OVM, Baremetal CloudStack orchestrates hypervisors and network appliances to give simple control to complex tasks through API or web GUI. (Yes, it’s like OpenStack) What is CloudStack
- 8. @CloudyAngus Think ‘your own Amazon Web Services’ Public clouds (SPs/MSPs) General public can create or log into instances themselves Private Clouds (Enterprises) Anyone who wants to be able to orchestrate their environment Hybrid Clouds (Enterprises) Balance/share load between their own DC and a Public Cloud Who uses CloudStack
- 9. @CloudyAngus This is a UI (for Linux guys who’ve forgotten). What is CloudStack
- 10. @CloudyAngus It’s OK it has an API cloudmonkey create network name=DemoNet displaytext=DemoNet zoneid=$ZONEID networkofferingid=$NEWORKOFFERINGID account=$ACCOUNTNAME domainid=$DOMAINID cloudmonkey list networks domainid="$DOMAINID" filter=id,name | jq '.network[] | select(.name=="DemoNet") | .id‘ cloudmonkey deploy virtualmachine networkids="$NETWORKID" serviceofferingid="$SERVICEOFFERINGID" templateid="$TEMPLATEID" zoneid="$ZONEID" account="$ACCOUNTNAME" domainid="$DOMAINID" startvm=true displayname=HTTPServer1 displayvm=true name=HTTPServer1 What is CloudStack
- 11. @CloudyAngus Push Infrastructure with Ansible Why? (Duuurrr)
- 12. @CloudyAngus CEO Why “Building CloudStack environments using Ansible? Are you just having fun with Ansible?”
- 13. @CloudyAngus Why Talented Cloud Architect “Noooo, if we can automate the building of environments using a powerful, simple and agentless technology we can make building at scale easy while ensuring that our results are consistent and repeatable.” “Building CloudStack environments using Ansible? Are you just having fun with Ansible?”
- 14. @CloudyAngus Why CEO “That would be excellent. Go ahead. Oh, and here’s a pay rise” “Noooo, if we can automate the building of environments using a powerful, simple and agentless technology we can make building at scale easy while ensuring that our results are consistent and repeatable.”
- 15. @CloudyAngus Some of that might actually have happened. Disclaimer Why
- 16. @CloudyAngus CSForge™ v CSForge delivers the rapid deployment of a standardised CloudStack powered IaaS cloud for small production deployments, or medium scale POCs or pilots. The framework can be used as a basis for public cloud or enterprise private cloud deployments Production v Cloud-‐scale environments initial deployment typically 24 – 100s of hosts v Often multi-‐hypervisor Why
- 17. @CloudyAngus Test/Dev Need to be able to create full environments to test: v CloudStack release candidates v CloudStack features v ShapeBlue patches Why
- 18. @CloudyAngus Why Ansible v Technical: v Client/Server architecture not required v Only SSH connectivity required (password or public/private keys) v …making it easier to build virgin environments v Modules can be in any language capable of returning JSON or key=value text pairs v Has an API v User: v Much shallower learning curve v Don’t need to learn a programming language (i.e. Ruby) v Not as many pre-‐existing playbooks (recipes/manifests) about, but improving with Ansible Galaxy
- 19. @CloudyAngus Typical Logical Production Topology Management (1Gb) CIMC (1GB) CIMC (1GB) Management (1GB LACP Bond) WWW Compute Hosts Storage Nodes Guest Public Storage (10GB LACP Bond) ManagementHosts Management 1GB Active/Passive Bond Load Balancers DNS/NTP Servers MySQL Master & Slave ACS Management Servers Deployment Server Management (1GB Active/Passive Bond) Storage (10GB LACP Bond) iDRAC (1GB) Storage (10GB LACP Bond) Public Cloud Compute Hosts Storage Link to ACS Managers
- 20. @CloudyAngus v 3 zones v 2 geographic locations v Upgrade done, then tests run for a week. Then VRs restarted Client Test Environment CCP3.0.7B MySQL CPBM 2.2 MySQL ESXi 1a ESXi 1b ESXi 1c vCenter Appliance ESXi 2a ESXi 2b ESXi 2c vCenter Appliance Zone 1 (local) Zone 2 (local) Zone 3 (remote) NFS NFS VPN VPN ESXi 2a ESXi 2b ESXi 2c vCenter Appliance NFS
- 21. @CloudyAngus Using Ansible to Deploy CloudStack Environments How? (variables, I Love ‘em)
- 22. @CloudyAngus # Copyright (C) ShapeBlue Ltd -‐ All Rights Reserved # Unauthorized copying of this file, via any medium is strictly prohibited # Proprietary and confidential # Released by ShapeBlue <info@shapeblue.com>, April 2014 -‐-‐-‐ -‐ name: Ensure selinux python bindings are installed yum: name=libselinux-‐python state=present -‐ name: Ensure the Apache Cloudstack Repo file is configured template: src=cloudstack.repo.j2 dest=/etc/yum.repos.d/cloudstack.repo -‐ name: Ensure selinux is to permissive command: setenforce permissive changed_when: false -‐ name: Ensure selinux is set permanently selinux: policy=targeted state=permissive -‐ name: Ensure CloudStack packages are installed yum: name=cloudstack-‐management state=present -‐ name: Ensure MySQL Client is present yum: name=mysql state=present -‐ name: Ensure vhd-‐util is present get_url: url="{{ vhdutil_url }}" dest=/usr/share/cloudstack-‐common/ scripts/vm/hypervisor/xenserver/vhd-‐util mode=0755 -‐ name: Ensure CloudStack Usage Service is installed yum: name=cloudstack-‐usage state=present -‐ name: Ensure CloudStack Usage Service is started service: name=cloudstack-‐usage state=started -‐ include: ./../galera-‐cluster/tasks/main.yml when: "{{ db_type }} == 'galera'" -‐ include: ./setupdb.yml CloudStack Management Server Role
- 23. @CloudyAngus # Copyright (C) ShapeBlue Ltd -‐ All Rights Reserved # Unauthorized copying of this file, via any medium is strictly prohibited # Proprietary and confidential # Released by ShapeBlue <info@shapeblue.com>, April 2014 [cloudstack] name=cloudstack baseurl=http://{{ acs_build_repo }}/{{ acs_build_path }} enabled=1 gpgcheck=1 cloudstack.repo.j2
- 24. @CloudyAngus # ansible-‐playbook management_vms_build.yml -‐i acs_demo_hosts -‐-‐ extra-‐vars "acs_build_path=tescng/centos/master" Playbook Arguments
- 25. @CloudyAngus Create bare VM + yum install git git clone mega repo with roles etc. Install & configure Ansible (included in repo) Update hosts and group_vars Create Deployment server (from role) locally PXE boot hosts/ mgmt VMs to bare OS Push application configuration to VMs and Hosts Building Environments
- 26. @CloudyAngus Distributed Architecture TEST STAGE PROD Deployment Srvr with cloned repo Deployment Srvr with cloned repo Deployment Srvr with cloned repo
- 27. @CloudyAngus v PXE VM against Ansible created KickStart file v DNS Servers (Bind 9) v Galera Cluster (Maria DB10 – any size) v MySQL Master/Slave v Clustered CloudStack Management Servers v HA Proxy (HA pair with keepalived) Management Infrastructure Roles
- 28. @CloudyAngus v PXE host against Ansible created KickStart or Answer file v KVM v Straight configuration of networking and packages v XenServer v XE commands via shell v Use ‘facts’ to determine installed patches v vSphere v 99% configured through kickstart file v SSH enabled (through KickStart) Hypervisor Host Roles
- 29. @CloudyAngus Using Ansible to Deploy CloudStack Environments Snippets
- 30. @CloudyAngus [management_hosts:children] cloudstack_manager_hosts loadbalancer_hosts deployment_hosts db_hosts [kvm_hosts:children] pod1_cluster1 pod1_cluster2 [pod1_cluster1] 10.101.1.13 pxemac=00:1c:c0:ec:b5:bc hostname=kvm-‐p1-‐c1-‐1 mgmt_ip=10.101.1.13 mgmt_netmask=255.255.255.0 Host File
- 31. @CloudyAngus Host File [mysql_master_host] 10.101.1.6 pxemac=00:50:56:8E:19:20 hostname=mysqlmasterhost mgmt_ip=10.101.1.6 mgmt_netmask=255.255.255.0 serverid=1 [pod1_cluster1] 10.101.1.13 pxemac=00:1c:c0:ec:b5:bc hostname=kvm-‐p1-‐c1-‐1 mgmt_ip=10.101.1.13 10.101.1.14 pxemac=00:1c:c0:ec:b5:43 hostname=kvm-‐p1-‐c1-‐2 mgmt_ip=10.101.1.14 Creating PXE files
- 32. @CloudyAngus File format: AA:BB:CC:DD:EE:FF must be 01-‐aa-‐bb-‐cc-‐dd-‐ee-‐ff -‐ name: create centos pxeconfig files template: src=pxelinuxcfg-‐centos6.j2 dest=/ptpboot/pxelinux.cfg/01-‐{{ hostvars[item]['pxemac'] | lower | replace(":","-‐") }} with_items: -‐ "{{ groups.management_hosts }}“ -‐ name: create KVM pxeconfig files template: src=pxelinuxcfg-‐kvm.j2 dest=/ptpboot/pxelinux.cfg/01-‐{{ hostvars[item]['pxemac'] | lower | replace(":","-‐") }} with_items: -‐ "{{ groups.kvm_hosts }}" Creating PXE files
- 33. @CloudyAngus # Ansible built pxeconfig script DEFAULT menu.c32 menu ctle ShapeBlue CSForge Deployment cmeout 10 LABEL {{ centos_iso_version }}-‐KVM kernel htp://{{ hostvars[inventory_hostname]['mgmt_ip'] }}/{{ centos_iso_version }}/ isolinux/vmlinuz append initrd=htp://{{ hostvars[inventory_hostname]['mgmt_ip'] }}/ {{ centos_iso_version }}/isolinux/initrd.img ramdisk_size=15491 ip=dhcp ksdevice={{ ks_device }} ks=htp://{{ hostvars[inventory_hostname]['mgmt_ip'] }}/{{ pxe_files_dir }}/{{ kvm_ks_file }} MENU LABEL {{ centos_iso_version }} for KVM pxelinuxcfg-‐kvm.j2
- 34. @CloudyAngus # Specifies the keyboard layout keyboard {{ keyboard_lang }} # Used with an HTTP install to specify where the install files are located url -‐-‐url htp://{{ hostvars[inventory_hostname]['mgmt_ip'] }}/{{ centos_iso_version }} # Assign a stacc IP address upon first boot & set the hostname network -‐-‐onboot yes -‐-‐device {{ ks_device }} -‐-‐bootproto dhcp -‐-‐noipv6 # Set the root password rootpw {{ mgmt_root_password }} mkdir /root/.ssh curl htp://{{ hostvars[inventory_hostname]['mgmt_ip'] }}/{{ publickey_file_name }} >> /root/.ssh/authorized_keys kickstart.j2
- 35. @CloudyAngus opcon subnet-‐mask {{ dhcp_netmask }}; opcon routers {{ dhcp_router }}; opcon broadcast-‐address {{ dhcp_broadcast }}; opcon domain-‐name-‐servers {{ int_dns_servers | join(", ") }}; opcon domain-‐name "{{ dns_domain }}"; subnet {{ dhcp_subnet }} netmask {{ dhcp_netmask }} { range {{ dhcp_range_starcp }} {{ dhcp_range_endip }}; # KVM Hosts {% for host in groups['kvm_hosts'] %} host {{ hostvars[host]['hostname'] }} { hardware ethernet {{ hostvars[host]['pxemac'] | lower }}; fixed-‐address {{ hostvars[host]['mgmt_ip'] }}; } {% endfor %} Creating dhcpd.conf
- 36. @CloudyAngus v Reverse Zone – Format: 2 IN PTR blah1.domain.com. ; Address Records {% for host in groups['management_hosts'] %} {{ hostvars[host]['mgmt_ip']|split('.')[3] }} IN PTR {{ hostvars[host]['hostname'] }}. {% endfor %} {% for host in groups['xenserver_hosts'] %} {{ hostvars[host]['mgmt_ip']|split('.')[3] }} IN PTR {{ hostvars[host]['hostname'] }}. {% endfor %} DNS Entries into Zone File – Jinja2
- 37. @CloudyAngus v Custom Filter to Split string import re def split_string(string, seperator=' '): return string.split(seperator) def split_regex(string, seperator_patern): return re.split(seperator_patern, string) class FilterModule(object): ''' A filter to split a string into a list. ''' def filters(self): return { 'split' : split_string, 'split_regex' : split_regex, } Custom Filter
- 38. @CloudyAngus v ansible.cfg # set plugin path directories here, separate with colons accon_plugins = /usr/share/ansible_plugins/accon_plugins callback_plugins = /usr/share/ansible_plugins/callback_plugins conneccon_plugins = /usr/share/ansible_plugins/conneccon_plugins lookup_plugins = /usr/share/ansible_plugins/lookup_plugins vars_plugins = /usr/share/ansible_plugins/vars_plugins filter_plugins = /usr/share/ansible_plugins/filter_plugins:/CSForge/custom_plugins/filter_plugins Add path if required
- 39. @CloudyAngus v ACS_env_group_vars.yml db_type: galera # opcons: mysql, mysql-‐masterslave, galera v build_db_servers.yml -‐-‐-‐ -‐ hosts: db_hosts roles: -‐ { role: galera-‐cluster, when: "'{{ db_type }}' == 'galera'" } -‐ { role: mysql, when: "'{{ db_type }}' != 'galera'" } Conditional Roles
- 40. Using Ansible to Deploy CloudStack Environments Paul Angus Cloud Architect ShapeBlue paul.angus@shapeblue.com Twitter: @CloudyAngus