SlideShare a Scribd company logo

ApacheCon Miami / CCCNA17 Using KVM in CloudStack

Dag Sonstebo, profile picture
Dag Sonstebo

The document discusses using KVM as the hypervisor with CloudStack. It provides background on KVM, how it works with CloudStack, and guidelines for installing and configuring it on the host including: - KVM is included in the Linux kernel and provides hardware virtualization. It works with CloudStack since version 2.0. - When using KVM with CloudStack, Linux bridges or Open vSwitch can be used for network bridging between VMs. Configuration examples are provided for both. - Storage configuration using a dedicated VLAN is also demonstrated. - The document provides tips for ensuring the KVM and libvirt services are configured correctly on the host for CloudStack management.

Technology
1 of 41
Downloaded 11 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
The Cloud Specialists Using the KVM hypervisor in CloudStack Dag Sonstebo Cloud Architect / Senior Consultant dag.sonstebo@shapeblue.com Twitter: @dagsonstebo
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue A b o u t M e • Cloud Architect @ ShapeBlue • Background: • Cloud and virtualization architect with 19 years experience from the service provider, financial and manufacturing industries. • Specialize in: • Cloud infrastructure architecture and engineering. • Virtualization - VMware vSphere, Citrix XenServer, KVM. • Automation • Involved with CloudStack since version 2.1.
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue “ShapeBlue are expert builders of public & private clouds. They are the leading global CloudStack services company.” A b o u t S h a p e B l u e
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue S h a p e B l u e c u s t o m e r s
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue S h a p e B l u e c u s t o m e r s
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue S h a p e B l u e c u s t o m e r s
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue CloudStack and hypervisor choice
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue C l o u d S t a c k h y p e r v i s o r c h o i c e Hypervisor Pros Cons VMware • Feature rich • Mature • Native HA / DRS • Bare metal • Cost – capex. • IP consumption • Virtual Centre XenServer • Mature • HA • Disk chains • Poolmaster • No workload balancing KVM • Fast • Maturing. • Cost • Non-proprietary • Lacking some networking, storage and snapshotting features. • No workload balancing
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue KVM background
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue K V M b a c k g r o u n d • KVM = Kernel-based Virtual Machine. • In CloudStack KVM is used as a type 2 hypervisor – provides virtualization support on top of a running Linux OS instance. • Included in the Linux kernel since version 2.6.20 (2007). • KVM is Free Software released under the GPL. • Supported in CloudStack since early cloud.com days (version 2.0).
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue K V M b a c k g r o u n d • Hardware: • 32 / 64 bit with CPU virtualization support (Intel-VTX / AMD-V). • Kernel modules • kvm • kvm_intel • kvm_amd • Para-virtualization through the Virtio API. • KVM and Qemu: • KVM abstracts access to the CPU and memory. • QEMU emulates the hardware resources (disks, graphics, USB, etc).
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue K V M a n d C l o u d S t a c k • Pros: • Flexible - not proprietary. • Fast. • Lightweight. • No SPOF – no poolmasters or VirtualCentre like components. • Cons: • No workload balancing (yet…). • No network throttling. • No VM snapshot support. • No native clustered file system. • Root volume migration requires VM to be shut down.
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Installation and configuration
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue B a s e i n s t a l l a t i o n • Host OS: • CentOS / RHEL 6.x / 7.x • Ubuntu 12.04 / 14.04 / 16.04 • 4GB memory / 64-bit X86. • Components (CloudStack 4.9): • NTP + DNS • Libvirt 1.2.0 or higher • Qemu / KVM: 2.0 or higher • Installing the CloudStack agent installs KVM and libvirt: • yum install cloudstack-agent • apt-get install cloudstack-agent
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue C o n f i g u r a t i o n • /etc/libvirt/libvirtd.conf: listen_tls = 0 listen_tcp = 1 tcp_port = "16509” auth_tcp = "none” mdns_adv = 0 • /etc/sysconfig/libvirtd: LIBVIRTD_ARGS="--listen” libvirtd_opts="-d -l” • Security policies: • CentOS: Selinux: permissive • Ubuntu: Apparmor • Firewall ports: • TCP/22 (SSH) • TCP/1798 (KVM) • TCP/16509 (Libvirt) • TCP/5900-6100 (VNC) • TCP/49152-49216 (Libvirt migration)
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Networking and storage
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue K V M n e t w o r k i n g • KVM utilizes network bridges – similar to Vmware vSwitches and XenServer networks. • Bridging back ends: • Linux bridge • Open Vswitch (OVS)
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue B r i d g e b a c k e n d s – L i n u x b r i d g e • Linux bridge • In Linux kernel since version 2.2. • Fast, simple, reliable and mature. • Provides L2 requirements for CloudStack. • Lacks automation options and scalable tunneling.
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue B r i d g e b a c k e n d s – O p e n V s w i t c h • Open vSwitch • Written for multi-host virtualization environments with dynamic end points, higher level of abstraction and potential for hardware offloading. • Network state kept in a network state database (OVSDB). • Allows for better automation. • SDN options (tunneling). • External controllers: OpenDaylight, Nicira, VMware NSX. • Default bridge backend in XenServer 6.0 and later versions (does not work with basic zones). • Some issues with VLAN configuration and stability (depending on host OS and version).
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue N e t w o r k i n g e x a m p l e • Cloudbr0: • Bond0: Eth0 + Eth1 • Carries management, guest and storage traffic. • Cloudbr1: • Bond1: Eth2 + Eth3 • Carries public traffic. • Cloud0: • Internal bridge, carries system VM management traffic. • Managed by CloudStack – does not need configuration.
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue L i n u x b r i d g e c o n f i g u r a t i o n ( C e n t O S ) ifcfg-eth0: DEVICE=eth0 TYPE=Ethernet USERCTL=no MASTER=bond0 SLAVE=yes BOOTPROTO=none NM_CONTROLLED=no ONBOOT=yes ifcfg-bond0: DEVICE=bond0 ONBOOT=yes BONDING_OPTS='mode=1 miimon=100' BRIDGE=cloudbr0 NM_CONTROLLED=no Ifcfg-cloudbr0: DEVICE=cloudbr0 ONBOOT=yes TYPE=Bridge IPADDR=192.168.100.20 NETMASK=255.255.255.0 GATEWAY=192.168.100.1 NM_CONTROLLED=no DELAY=0
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue L i n u x b r i d g e c o n f i g u r a t i o n ( C e n t O S ) ifcfg-cloudbr1: DEVICE=cloudbr1 ONBOOT=yes TYPE=Bridge NM_CONTROLLED=no DELAY=0
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue L i n u x b r i d g e c o n f i g u r a t i o n ( C e n t O S ) Storage VLAN ifcfg-bond.100: DEVICE=bond0.100 VLAN=yes BOOTPROTO=static ONBOOT=yes TYPE=Unknown BRIDGE=cloudbr100 Storage VLAN ifcfg-cloudbr100: DEVICE=cloudbr100 ONBOOT=yes TYPE=Bridge VLAN=yes IPADDR=10.0.100.20 NETMASK=255.255.255.0 NM_CONTROLLED=no DELAY=0
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue L i n u x b r i d g e c o n f i g u r a t i o n ( C e n t O S ) • Check bridges: # brctl show bridge name bridge id STP enabled interfaces brbond0-113 8000.000c29b43c4d no bond0.113 cloudbr0 8000.000c29b55932 no bond0 cloudbr1 8000.000c29b45956 no bond1 cloudbr100 8000.000c29b43c4d no bond0.100 • Check bonds: # cat /proc/net/bonding/bond0 Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011) Bonding Mode: fault-tolerance (active-backup) Primary Slave: None Currently Active Slave: eth0 MII Status: up MII Polling Interval (ms): 100 Up Delay (ms): 0 Down Delay (ms): 0 Slave Interface: eth0 MII Status: up
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue O V S c o n f i g u r a t i o n ( C e n t O S ) • Requires OVS installation: # apt-get install openvswitch-switch # yum install openvswitch-<version>.rpm # yum install openvswitch-kmod-<version>.rpm • Add bridges and bonds with OVS command line tools: # ovs-vsctl add-br cloudbr0 # ovs-vsctl add-br cloudbr1 # ovs-vsctl add-bond cloudbr0 bond0 eth0 eth1 # ovs-vsctl add-bond cloudbr1 bond1 eth2 eth3
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue O V S c o n f i g u r a t i o n ( C e n t O S ) ifcfg-eth0: DEVICE=eth0 ONBOOT=yes NM_CONTROLLED=no BOOTPROTO=none HOTPLUG=no ifcfg-bond0: DEVICE=bond0 ONBOOT=yes DEVICETYPE=ovs TYPE=OVSBond OVS_BRIDGE=cloudbr0 BOOTPROTO=none BOND_IFACES="eth0 eth1" OVS_OPTIONS="bond_mode= active-backup lacp=off other_config:bond- detect-mode=miimon other_config:bond- miimon-interval=100" HOTPLUG=no Ifcfg-cloudbr0: DEVICE=cloudbr0 ONBOOT=yes DEVICETYPE=ovs TYPE=OVSBridge BOOTPROTO=static IPADDR=192.168.100.20 NETMASK=255.255.255.0 HOTPLUG=no
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue O V S c o n f i g u r a t i o n ( C e n t O S ) ifcfg-cloud0: DEVICE=cloud0 ONBOOT=yes DEVICETYPE=ovs TYPE=OVSBridge BOOTPROTO=static HOTPLUG=no IPADDR=169.254.0.1 NETMASK=255.255.0.0 VLAN configuration: # ovs-vsctl add-br cloudbr100 cloudbr0 100 ifcfg-cloudbr100 DEVICE=cloudbr100 ONBOOT=yes DEVICETYPE=ovs TYPE=OVSBridge BOOTPROTO=static IPADDR=10.0.100.20 NETMASK=255.255.255.0 HOTPLUG=no
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue O V S c o n f i g u r a t i o n ( C e n t O S ) • Some issues with network drivers cause VLANs to not propagate: # ovs-vsctl set interface eth0 other-config:enable-vlan- splinters=true • List bridges: # ovs-vsctl show 27daed4e-52f3-4177-9827-550f0e7df452 Bridge "cloudbr1" Port "vnet2" Interface "vnet2" Port "bond1" Interface "eth3" Interface "eth2" Port "cloudbr1" Interface "cloudbr1" type: internal Bridge "cloud0" … • List bonds: ovs-appctl bond/show bond0 ---- bond0 ---- bond_mode: active-backup bond may use recirculation: no, Recirc-ID : -1 bond-hash-basis: 0 updelay: 0 ms downdelay: 0 ms lacp_status: off active slave mac: 00:0c:xx:xx:xx:xx(eth0) slave eth0: enabled active slave may_enable: true slave eth1: enabled may_enable: true
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue O V S c o n f i g u r a t i o n ( C e n t O S ) • Agent properties file - /etc/cloudstack/agent/agent.properties: guest.network.device=cloudbr0 workers=5 private.network.device=cloudbr0 network.bridge.type=openvswitch port=8250 resource=com.cloud.hypervisor.kvm.resource.LibvirtComputingResource pod=3 zone=2 hypervisor.type=kvm guid=c3c6a2fc-13d3-3af1-ae2e-e48a21d9b883 public.network.device=cloudbr1 cluster=5 local.storage.uuid=2f2220ef-2624-4e69-b442-0a4ae5c5add6 domr.scripts.dir=scripts/network/domr/kvm LibvirtComputingResource.id=28 host=192.168.100.20
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue S t o r a g e • Disks, templates and snapshots use QCOW2 format. • Primary storage options: • NFS – recommended if performance is sufficient, easy option, supports CloudStack managed thin provisioning. • Local storage (does not support storage migration). • SharedMountpoint option used for iSCSI / FC. • CEPH. • SharedMountpoint: • Used to access pre-defined block storage. • Must be running a clustered file system. • Preconfigured on each host. • Consistent – same on each host – e.g. /mnt/primary
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue S t o r a g e – s h a r e d m o u n t p o i n t • KVM lacks a native clustered file system like VMFS or SR (CLVM). • Clustered file system options: • OCFS2 (Oracle) • GFS2 (RedHat) • CLVM (not officially supported in CloudStack) • CEPH
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Management and troubleshooting
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue H o w d o I m a n a g e K V M • Countless tools (40+). • Most common: • Virsh command line tool: • Virt-manager: linux native but works well with SSH X session forwarding. • Configuration management: • Anything that utilises libvirt. • Ansible: virt modules • Puppet modules and Chef Cookbooks. # ssh –X kvmhost1 root@kvmhost1:~# virsh list Id Name State ----------------------------------- 2 r-540-VM running 3 s-548-VM running 4 v-509-VM running root@kvmhost1:~# virt-manager &
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Tr o u b l e s h o o t i n g • Check KVM is running: # lsmod | grep kvm kvm_intel 151552 9 kvm 479232 1 kvm_intel • Log file: /var/log/cloudstack/agent/agent.log • Increase logging verbosity: # sed -i 's/INFO/DEBUG/g' /etc/cloudstack/agent/log4j-cloud.xml • KVM uses link local networking, hence connect to system VMs using: # ssh -i /root/.ssh/id_rsa.cloud -p 3922 root@<linklocalIP>
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue W h a t ’ s n e x t • HA (and IPMI control) is being developed for KVM. https://cwiki.apache.org/confluence/display/CLOUDSTACK/KV M+HA+with+IPMI+Fencing • DRS functionality has been discussed but is not yet in development.
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Questions?
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue M o r e i n f o r m a t i o n • Background: • http://www.linux-kvm.org • http://wiki.qemu.org/KVM • https://libvirt.org/ • Installation: • http://docs.cloudstack.apache.org/projects/cloudstack- installation/en/4.9/hypervisor/kvm.html • Management tools: • http://www.linux-kvm.org/page/Management_Tools
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue M o r e i n f o r m a t i o n • Networking: • http://openvswitch.org • http://openvswitch.org/support/dist-docs/WHY-OVS.md.txt • Storage: • CEPH: http://docs.ceph.com/docs/hammer/rbd/rbd-cloudstack/ • OCFS2: https://oss.oracle.com/projects/ocfs2/ • GFS2: https://access.redhat.com/documentation/en- US/Red_Hat_Enterprise_Linux/6/html- single/Global_File_System_2/index.html
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue M o r e i n f o r m a t i o n • Slide deck: https://www.slideshare.net/ShapeBlue/ccna17-kvm- and-cloudstack • Blog: http://shapeblue.com/blog http://dsonstebo.wordpress.com • Email: dag.sonstebo@shapeblue.com • Twitter: @dagsonstebo • Web: http://shapeblue.com
C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue S h a m e l e s s s e l f p r o m o t i o n … … • My next talk: • What: Apache CloudStack Upgrade Best Practices • When: Tomorrow – Thursday @ 2:40PM • Where: Rafael (here….)
The Cloud Specialists Using the KVM hypervisor in CloudStack Dag Sonstebo Cloud Architect / Senior Consultant dag.sonstebo@shapeblue.com Twitter: @dagsonstebo

More Related Content

PDF
ApacheCon Miami / CCCNA17 CloudStack upgrade best practices
Dag Sonstebo
 
PDF
TechUG Glasgow talk 22/Feb/17 Configuration Management Best Practices
Dag Sonstebo
 
PDF
CCNA17 KVM and CloudStack
ShapeBlue
 
PDF
CCCNA17 CloudStack upgrade best practices
ShapeBlue
 
PDF
CCCNA17 Distributed CI and Testing for Cloudstack in a Hybrid Community
ShapeBlue
 
PDF
CCCNA17 Introduction
ShapeBlue
 
PDF
CCCNA17 Reliable Host Fencing
ShapeBlue
 
PPTX
CCCNA17 CloudStack Container Service
ShapeBlue
 
ApacheCon Miami / CCCNA17 CloudStack upgrade best practices
Dag Sonstebo
 
TechUG Glasgow talk 22/Feb/17 Configuration Management Best Practices
Dag Sonstebo
 
CCNA17 KVM and CloudStack
ShapeBlue
 
CCCNA17 CloudStack upgrade best practices
ShapeBlue
 
CCCNA17 Distributed CI and Testing for Cloudstack in a Hybrid Community
ShapeBlue
 
CCCNA17 Introduction
ShapeBlue
 
CCCNA17 Reliable Host Fencing
ShapeBlue
 
CCCNA17 CloudStack Container Service
ShapeBlue
 

What's hot (20)

PDF
CloudStack upgrade best practices - Dag Sonstebo
Ingo Jochim
 
PDF
CCCNA17 Dynamic Roles in CloudStack
ShapeBlue
 
PPTX
CCNA17 CloudStack and NFV
ShapeBlue
 
PDF
CloudStack UI
ShapeBlue
 
PPTX
Dynamic roles in cloudstack
ShapeBlue
 
PDF
CloudStack and NFV
ShapeBlue
 
PPTX
CloudStack EU user group - Trillian
ShapeBlue
 
PDF
Paul Angus - CloudStack Backup and Recovery Framework
ShapeBlue
 
PPTX
Using the KVMhypervisor in CloudStack
ShapeBlue
 
PDF
CloudStack and testing
ShapeBlue
 
PDF
Giles Sirett: Introduction and CloudStack news
ShapeBlue
 
PDF
New stuff in CloudStack!
ShapeBlue
 
PDF
CloudStack usage service
ShapeBlue
 
PPTX
Introduction and CloudStack news
ShapeBlue
 
PDF
Boris Stoyanov - some new features in Apache cloudStack
ShapeBlue
 
PDF
Paul Angus: CloudStack new UI (Primate)
ShapeBlue
 
PPTX
Monkey man
ShapeBlue
 
PDF
Paul Angus – Backup & Recovery in CloudStack
ShapeBlue
 
PDF
CSEUG introduction
ShapeBlue
 
PPTX
CloudStack News, Berlin 16 june 2016
ShapeBlue
 
CloudStack upgrade best practices - Dag Sonstebo
Ingo Jochim
 
CCCNA17 Dynamic Roles in CloudStack
ShapeBlue
 
CCNA17 CloudStack and NFV
ShapeBlue
 
CloudStack UI
ShapeBlue
 
Dynamic roles in cloudstack
ShapeBlue
 
CloudStack and NFV
ShapeBlue
 
CloudStack EU user group - Trillian
ShapeBlue
 
Paul Angus - CloudStack Backup and Recovery Framework
ShapeBlue
 
Using the KVMhypervisor in CloudStack
ShapeBlue
 
CloudStack and testing
ShapeBlue
 
Giles Sirett: Introduction and CloudStack news
ShapeBlue
 
New stuff in CloudStack!
ShapeBlue
 
CloudStack usage service
ShapeBlue
 
Introduction and CloudStack news
ShapeBlue
 
Boris Stoyanov - some new features in Apache cloudStack
ShapeBlue
 
Paul Angus: CloudStack new UI (Primate)
ShapeBlue
 
Monkey man
ShapeBlue
 
Paul Angus – Backup & Recovery in CloudStack
ShapeBlue
 
CSEUG introduction
ShapeBlue
 
CloudStack News, Berlin 16 june 2016
ShapeBlue
 
Ad

Similar to ApacheCon Miami / CCCNA17 Using KVM in CloudStack (20)

PDF
Paul Angus - what's new in ACS 4.11
ShapeBlue
 
PDF
Whats new in Cloudstack 4.11 - behind the headlines
ShapeBlue
 
PPTX
Building clouds with apache cloudstack apache roadshow 2018
ShapeBlue
 
PPTX
Webinar: Agile Network Deployment
VasudhaSridharan
 
PPTX
Paul Angus - CloudStack Container Service
ShapeBlue
 
PPSX
CloudStack + KVM: Your Local Cloud Lab
ShapeBlue
 
PDF
K8s is not for App Developers
QAware GmbH
 
PDF
BRKDCN-2613.pdf Cisco AI DC NVIDIA presentation
demidovs1
 
PPT
Enhancing Computer Security via End-to-End Communication Visualization
amiable_indian
 
PDF
SDN Presentation
Abderrahmane TEKFI
 
PDF
Puppet Camp London Fall 2015 - Service Discovery and Puppet
Marc Cluet
 
PDF
London Puppet Camp 2015: Service Discovery and Puppet
Puppet
 
PDF
NSO: Network Service Orchestrator enabled by Tail-f Hands-on Lab
Cisco Canada
 
PDF
OSGi Remote Services - Alexander Broekhuis, Bram de Kruijff
mfrancis
 
PDF
DEVNET-1148 Leveraging Cisco OpenStack Private Cloud for Developers
Cisco DevNet
 
PDF
CERN OpenStack Cloud Control Plane - From VMs to K8s
Belmiro Moreira
 
PDF
Zombies in Kubernetes
Thomas Fricke
 
PPTX
Stups.io - an Open Source Cloud Framework for AWS
Jan Löffler
 
PDF
Container Camp London (2016-09-09)
craigbox
 
PDF
Manage your switches like servers
Cumulus Networks
 
Paul Angus - what's new in ACS 4.11
ShapeBlue
 
Whats new in Cloudstack 4.11 - behind the headlines
ShapeBlue
 
Building clouds with apache cloudstack apache roadshow 2018
ShapeBlue
 
Webinar: Agile Network Deployment
VasudhaSridharan
 
Paul Angus - CloudStack Container Service
ShapeBlue
 
CloudStack + KVM: Your Local Cloud Lab
ShapeBlue
 
K8s is not for App Developers
QAware GmbH
 
BRKDCN-2613.pdf Cisco AI DC NVIDIA presentation
demidovs1
 
Enhancing Computer Security via End-to-End Communication Visualization
amiable_indian
 
SDN Presentation
Abderrahmane TEKFI
 
Puppet Camp London Fall 2015 - Service Discovery and Puppet
Marc Cluet
 
London Puppet Camp 2015: Service Discovery and Puppet
Puppet
 
NSO: Network Service Orchestrator enabled by Tail-f Hands-on Lab
Cisco Canada
 
OSGi Remote Services - Alexander Broekhuis, Bram de Kruijff
mfrancis
 
DEVNET-1148 Leveraging Cisco OpenStack Private Cloud for Developers
Cisco DevNet
 
CERN OpenStack Cloud Control Plane - From VMs to K8s
Belmiro Moreira
 
Zombies in Kubernetes
Thomas Fricke
 
Stups.io - an Open Source Cloud Framework for AWS
Jan Löffler
 
Container Camp London (2016-09-09)
craigbox
 
Manage your switches like servers
Cumulus Networks
 
Ad

Recently uploaded (20)

PPTX
Group 1 Presentation -Planning and Decision Making .pptx
MatthewLewis227954
 
PDF
Assigned Numbers - 2025 - Bluetooth® Document
Bloombase
 
PDF
Unlocking AI with Model Context Protocol (MCP)
Brian McKeiver
 
PDF
Machine learning based COVID-19 study performance prediction
IAESIJAI
 
PPTX
A Presentation on Artificial Intelligence
memembruh336
 
PDF
Electronic commerce courselecture one. Pdf
OmerMohamed64
 
PPTX
1. Introduction to Computer Programming.pptx
bonakiduza1024
 
PPTX
Programs and apps: productivity, graphics, security and other tools
4mqw9zch22
 
PDF
Optimiser vos workloads AI/ML sur Amazon EC2 et AWS Graviton
Julien SIMON
 
PDF
Reach Out and Touch Someone: Haptics and Empathic Computing
Mark Billinghurst
 
PDF
The Rise and Fall of 3GPP – Time for a Sabbatical?
3G4G
 
PDF
A comparative analysis of optical character recognition models for extracting...
IAESIJAI
 
PDF
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
Jian-Hong Pan
 
PDF
gpt5_lecture_notes_comprehensive_20250812015547.pdf
Chinchu40
 
PDF
Building Integrated photovoltaic BIPV_UPV.pdf
SandeepSingh286037
 
PDF
Approach and Philosophy of On baking technology
30anthuong17
 
PDF
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
SOFTTECHHUB
 
PDF
Empathic Computing: Creating Shared Understanding
Mark Billinghurst
 
PPTX
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
muhammadmuneebnaeem7
 
PPTX
MYSQL Presentation for SQL database connectivity
Swati270511
 
Group 1 Presentation -Planning and Decision Making .pptx
MatthewLewis227954
 
Assigned Numbers - 2025 - Bluetooth® Document
Bloombase
 
Unlocking AI with Model Context Protocol (MCP)
Brian McKeiver
 
Machine learning based COVID-19 study performance prediction
IAESIJAI
 
A Presentation on Artificial Intelligence
memembruh336
 
Electronic commerce courselecture one. Pdf
OmerMohamed64
 
1. Introduction to Computer Programming.pptx
bonakiduza1024
 
Programs and apps: productivity, graphics, security and other tools
4mqw9zch22
 
Optimiser vos workloads AI/ML sur Amazon EC2 et AWS Graviton
Julien SIMON
 
Reach Out and Touch Someone: Haptics and Empathic Computing
Mark Billinghurst
 
The Rise and Fall of 3GPP – Time for a Sabbatical?
3G4G
 
A comparative analysis of optical character recognition models for extracting...
IAESIJAI
 
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
Jian-Hong Pan
 
gpt5_lecture_notes_comprehensive_20250812015547.pdf
Chinchu40
 
Building Integrated photovoltaic BIPV_UPV.pdf
SandeepSingh286037
 
Approach and Philosophy of On baking technology
30anthuong17
 
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
SOFTTECHHUB
 
Empathic Computing: Creating Shared Understanding
Mark Billinghurst
 
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
muhammadmuneebnaeem7
 
MYSQL Presentation for SQL database connectivity
Swati270511
 

ApacheCon Miami / CCCNA17 Using KVM in CloudStack

  • 2. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue A b o u t M e • Cloud Architect @ ShapeBlue • Background: • Cloud and virtualization architect with 19 years experience from the service provider, financial and manufacturing industries. • Specialize in: • Cloud infrastructure architecture and engineering. • Virtualization - VMware vSphere, Citrix XenServer, KVM. • Automation • Involved with CloudStack since version 2.1.
  • 3. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue “ShapeBlue are expert builders of public & private clouds. They are the leading global CloudStack services company.” A b o u t S h a p e B l u e
  • 4. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue S h a p e B l u e c u s t o m e r s
  • 5. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue S h a p e B l u e c u s t o m e r s
  • 6. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue S h a p e B l u e c u s t o m e r s
  • 7. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue CloudStack and hypervisor choice
  • 8. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue C l o u d S t a c k h y p e r v i s o r c h o i c e Hypervisor Pros Cons VMware • Feature rich • Mature • Native HA / DRS • Bare metal • Cost – capex. • IP consumption • Virtual Centre XenServer • Mature • HA • Disk chains • Poolmaster • No workload balancing KVM • Fast • Maturing. • Cost • Non-proprietary • Lacking some networking, storage and snapshotting features. • No workload balancing
  • 9. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue KVM background
  • 10. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue K V M b a c k g r o u n d • KVM = Kernel-based Virtual Machine. • In CloudStack KVM is used as a type 2 hypervisor – provides virtualization support on top of a running Linux OS instance. • Included in the Linux kernel since version 2.6.20 (2007). • KVM is Free Software released under the GPL. • Supported in CloudStack since early cloud.com days (version 2.0).
  • 11. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue K V M b a c k g r o u n d • Hardware: • 32 / 64 bit with CPU virtualization support (Intel-VTX / AMD-V). • Kernel modules • kvm • kvm_intel • kvm_amd • Para-virtualization through the Virtio API. • KVM and Qemu: • KVM abstracts access to the CPU and memory. • QEMU emulates the hardware resources (disks, graphics, USB, etc).
  • 12. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue K V M a n d C l o u d S t a c k • Pros: • Flexible - not proprietary. • Fast. • Lightweight. • No SPOF – no poolmasters or VirtualCentre like components. • Cons: • No workload balancing (yet…). • No network throttling. • No VM snapshot support. • No native clustered file system. • Root volume migration requires VM to be shut down.
  • 13. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Installation and configuration
  • 14. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue B a s e i n s t a l l a t i o n • Host OS: • CentOS / RHEL 6.x / 7.x • Ubuntu 12.04 / 14.04 / 16.04 • 4GB memory / 64-bit X86. • Components (CloudStack 4.9): • NTP + DNS • Libvirt 1.2.0 or higher • Qemu / KVM: 2.0 or higher • Installing the CloudStack agent installs KVM and libvirt: • yum install cloudstack-agent • apt-get install cloudstack-agent
  • 15. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue C o n f i g u r a t i o n • /etc/libvirt/libvirtd.conf: listen_tls = 0 listen_tcp = 1 tcp_port = "16509” auth_tcp = "none” mdns_adv = 0 • /etc/sysconfig/libvirtd: LIBVIRTD_ARGS="--listen” libvirtd_opts="-d -l” • Security policies: • CentOS: Selinux: permissive • Ubuntu: Apparmor • Firewall ports: • TCP/22 (SSH) • TCP/1798 (KVM) • TCP/16509 (Libvirt) • TCP/5900-6100 (VNC) • TCP/49152-49216 (Libvirt migration)
  • 16. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Networking and storage
  • 17. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue K V M n e t w o r k i n g • KVM utilizes network bridges – similar to Vmware vSwitches and XenServer networks. • Bridging back ends: • Linux bridge • Open Vswitch (OVS)
  • 18. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue B r i d g e b a c k e n d s – L i n u x b r i d g e • Linux bridge • In Linux kernel since version 2.2. • Fast, simple, reliable and mature. • Provides L2 requirements for CloudStack. • Lacks automation options and scalable tunneling.
  • 19. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue B r i d g e b a c k e n d s – O p e n V s w i t c h • Open vSwitch • Written for multi-host virtualization environments with dynamic end points, higher level of abstraction and potential for hardware offloading. • Network state kept in a network state database (OVSDB). • Allows for better automation. • SDN options (tunneling). • External controllers: OpenDaylight, Nicira, VMware NSX. • Default bridge backend in XenServer 6.0 and later versions (does not work with basic zones). • Some issues with VLAN configuration and stability (depending on host OS and version).
  • 20. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue N e t w o r k i n g e x a m p l e • Cloudbr0: • Bond0: Eth0 + Eth1 • Carries management, guest and storage traffic. • Cloudbr1: • Bond1: Eth2 + Eth3 • Carries public traffic. • Cloud0: • Internal bridge, carries system VM management traffic. • Managed by CloudStack – does not need configuration.
  • 21. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue L i n u x b r i d g e c o n f i g u r a t i o n ( C e n t O S ) ifcfg-eth0: DEVICE=eth0 TYPE=Ethernet USERCTL=no MASTER=bond0 SLAVE=yes BOOTPROTO=none NM_CONTROLLED=no ONBOOT=yes ifcfg-bond0: DEVICE=bond0 ONBOOT=yes BONDING_OPTS='mode=1 miimon=100' BRIDGE=cloudbr0 NM_CONTROLLED=no Ifcfg-cloudbr0: DEVICE=cloudbr0 ONBOOT=yes TYPE=Bridge IPADDR=192.168.100.20 NETMASK=255.255.255.0 GATEWAY=192.168.100.1 NM_CONTROLLED=no DELAY=0
  • 22. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue L i n u x b r i d g e c o n f i g u r a t i o n ( C e n t O S ) ifcfg-cloudbr1: DEVICE=cloudbr1 ONBOOT=yes TYPE=Bridge NM_CONTROLLED=no DELAY=0
  • 23. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue L i n u x b r i d g e c o n f i g u r a t i o n ( C e n t O S ) Storage VLAN ifcfg-bond.100: DEVICE=bond0.100 VLAN=yes BOOTPROTO=static ONBOOT=yes TYPE=Unknown BRIDGE=cloudbr100 Storage VLAN ifcfg-cloudbr100: DEVICE=cloudbr100 ONBOOT=yes TYPE=Bridge VLAN=yes IPADDR=10.0.100.20 NETMASK=255.255.255.0 NM_CONTROLLED=no DELAY=0
  • 24. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue L i n u x b r i d g e c o n f i g u r a t i o n ( C e n t O S ) • Check bridges: # brctl show bridge name bridge id STP enabled interfaces brbond0-113 8000.000c29b43c4d no bond0.113 cloudbr0 8000.000c29b55932 no bond0 cloudbr1 8000.000c29b45956 no bond1 cloudbr100 8000.000c29b43c4d no bond0.100 • Check bonds: # cat /proc/net/bonding/bond0 Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011) Bonding Mode: fault-tolerance (active-backup) Primary Slave: None Currently Active Slave: eth0 MII Status: up MII Polling Interval (ms): 100 Up Delay (ms): 0 Down Delay (ms): 0 Slave Interface: eth0 MII Status: up
  • 25. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue O V S c o n f i g u r a t i o n ( C e n t O S ) • Requires OVS installation: # apt-get install openvswitch-switch # yum install openvswitch-<version>.rpm # yum install openvswitch-kmod-<version>.rpm • Add bridges and bonds with OVS command line tools: # ovs-vsctl add-br cloudbr0 # ovs-vsctl add-br cloudbr1 # ovs-vsctl add-bond cloudbr0 bond0 eth0 eth1 # ovs-vsctl add-bond cloudbr1 bond1 eth2 eth3
  • 26. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue O V S c o n f i g u r a t i o n ( C e n t O S ) ifcfg-eth0: DEVICE=eth0 ONBOOT=yes NM_CONTROLLED=no BOOTPROTO=none HOTPLUG=no ifcfg-bond0: DEVICE=bond0 ONBOOT=yes DEVICETYPE=ovs TYPE=OVSBond OVS_BRIDGE=cloudbr0 BOOTPROTO=none BOND_IFACES="eth0 eth1" OVS_OPTIONS="bond_mode= active-backup lacp=off other_config:bond- detect-mode=miimon other_config:bond- miimon-interval=100" HOTPLUG=no Ifcfg-cloudbr0: DEVICE=cloudbr0 ONBOOT=yes DEVICETYPE=ovs TYPE=OVSBridge BOOTPROTO=static IPADDR=192.168.100.20 NETMASK=255.255.255.0 HOTPLUG=no
  • 27. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue O V S c o n f i g u r a t i o n ( C e n t O S ) ifcfg-cloud0: DEVICE=cloud0 ONBOOT=yes DEVICETYPE=ovs TYPE=OVSBridge BOOTPROTO=static HOTPLUG=no IPADDR=169.254.0.1 NETMASK=255.255.0.0 VLAN configuration: # ovs-vsctl add-br cloudbr100 cloudbr0 100 ifcfg-cloudbr100 DEVICE=cloudbr100 ONBOOT=yes DEVICETYPE=ovs TYPE=OVSBridge BOOTPROTO=static IPADDR=10.0.100.20 NETMASK=255.255.255.0 HOTPLUG=no
  • 28. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue O V S c o n f i g u r a t i o n ( C e n t O S ) • Some issues with network drivers cause VLANs to not propagate: # ovs-vsctl set interface eth0 other-config:enable-vlan- splinters=true • List bridges: # ovs-vsctl show 27daed4e-52f3-4177-9827-550f0e7df452 Bridge "cloudbr1" Port "vnet2" Interface "vnet2" Port "bond1" Interface "eth3" Interface "eth2" Port "cloudbr1" Interface "cloudbr1" type: internal Bridge "cloud0" … • List bonds: ovs-appctl bond/show bond0 ---- bond0 ---- bond_mode: active-backup bond may use recirculation: no, Recirc-ID : -1 bond-hash-basis: 0 updelay: 0 ms downdelay: 0 ms lacp_status: off active slave mac: 00:0c:xx:xx:xx:xx(eth0) slave eth0: enabled active slave may_enable: true slave eth1: enabled may_enable: true
  • 29. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue O V S c o n f i g u r a t i o n ( C e n t O S ) • Agent properties file - /etc/cloudstack/agent/agent.properties: guest.network.device=cloudbr0 workers=5 private.network.device=cloudbr0 network.bridge.type=openvswitch port=8250 resource=com.cloud.hypervisor.kvm.resource.LibvirtComputingResource pod=3 zone=2 hypervisor.type=kvm guid=c3c6a2fc-13d3-3af1-ae2e-e48a21d9b883 public.network.device=cloudbr1 cluster=5 local.storage.uuid=2f2220ef-2624-4e69-b442-0a4ae5c5add6 domr.scripts.dir=scripts/network/domr/kvm LibvirtComputingResource.id=28 host=192.168.100.20
  • 30. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue S t o r a g e • Disks, templates and snapshots use QCOW2 format. • Primary storage options: • NFS – recommended if performance is sufficient, easy option, supports CloudStack managed thin provisioning. • Local storage (does not support storage migration). • SharedMountpoint option used for iSCSI / FC. • CEPH. • SharedMountpoint: • Used to access pre-defined block storage. • Must be running a clustered file system. • Preconfigured on each host. • Consistent – same on each host – e.g. /mnt/primary
  • 31. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue S t o r a g e – s h a r e d m o u n t p o i n t • KVM lacks a native clustered file system like VMFS or SR (CLVM). • Clustered file system options: • OCFS2 (Oracle) • GFS2 (RedHat) • CLVM (not officially supported in CloudStack) • CEPH
  • 32. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Management and troubleshooting
  • 33. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue H o w d o I m a n a g e K V M • Countless tools (40+). • Most common: • Virsh command line tool: • Virt-manager: linux native but works well with SSH X session forwarding. • Configuration management: • Anything that utilises libvirt. • Ansible: virt modules • Puppet modules and Chef Cookbooks. # ssh –X kvmhost1 root@kvmhost1:~# virsh list Id Name State ----------------------------------- 2 r-540-VM running 3 s-548-VM running 4 v-509-VM running root@kvmhost1:~# virt-manager &
  • 34. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Tr o u b l e s h o o t i n g • Check KVM is running: # lsmod | grep kvm kvm_intel 151552 9 kvm 479232 1 kvm_intel • Log file: /var/log/cloudstack/agent/agent.log • Increase logging verbosity: # sed -i 's/INFO/DEBUG/g' /etc/cloudstack/agent/log4j-cloud.xml • KVM uses link local networking, hence connect to system VMs using: # ssh -i /root/.ssh/id_rsa.cloud -p 3922 root@<linklocalIP>
  • 35. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue W h a t ’ s n e x t • HA (and IPMI control) is being developed for KVM. https://cwiki.apache.org/confluence/display/CLOUDSTACK/KV M+HA+with+IPMI+Fencing • DRS functionality has been discussed but is not yet in development.
  • 36. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Questions?
  • 37. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue M o r e i n f o r m a t i o n • Background: • http://www.linux-kvm.org • http://wiki.qemu.org/KVM • https://libvirt.org/ • Installation: • http://docs.cloudstack.apache.org/projects/cloudstack- installation/en/4.9/hypervisor/kvm.html • Management tools: • http://www.linux-kvm.org/page/Management_Tools
  • 38. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue M o r e i n f o r m a t i o n • Networking: • http://openvswitch.org • http://openvswitch.org/support/dist-docs/WHY-OVS.md.txt • Storage: • CEPH: http://docs.ceph.com/docs/hammer/rbd/rbd-cloudstack/ • OCFS2: https://oss.oracle.com/projects/ocfs2/ • GFS2: https://access.redhat.com/documentation/en- US/Red_Hat_Enterprise_Linux/6/html- single/Global_File_System_2/index.html
  • 39. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue M o r e i n f o r m a t i o n • Slide deck: https://www.slideshare.net/ShapeBlue/ccna17-kvm- and-cloudstack • Blog: http://shapeblue.com/blog http://dsonstebo.wordpress.com • Email: dag.sonstebo@shapeblue.com • Twitter: @dagsonstebo • Web: http://shapeblue.com
  • 40. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue S h a m e l e s s s e l f p r o m o t i o n … … • My next talk: • What: Apache CloudStack Upgrade Best Practices • When: Tomorrow – Thursday @ 2:40PM • Where: Rafael (here….)