Intro to OpenStack - WAJUG

Rackspace Technical Services
Introduction to OpenStack
May 8, 2014
WAJUG Meetup

About.me/kevjackson
RACKSPACE | www.rackspace.com
2
Kevin Jackson, Principal Architect
@itarchitectkev

RACKSPACE® HOSTING | WWW.RACKSPACE.COM
OpenStack

OpenStack is open-source software used to build public, private
and hybrid clouds
16,200 138
COMMUNITYSOFTWARE
Join our global community of technologists,
developers, researchers, corporations and
cloud computing experts.
OpenStack Software delivers a massively
scalable cloud operating system.
COMPUTE
NETWORKIN
G
STORAGE
PEOPLE COUNTRIES
4
What is OpenStack?

The Mission
5
“To produce the ubiquitous Open Source Cloud
Computing platform that will meet the needs of
public and private clouds regardless of size, by
being simple to implement and massively
scalable.”

KEYSTONE
IDENTITY
GLANCE
IMAGE MANAGEMENT
NOVA
COMPUTE LAYER
SWIFT
OBJECT STORE
CINDER
BLOCK STORAGE
NEUTRON
NETWORKING
HORIZON
DASHBOARD
CEILOMETER
TELEMETRY
HEAT
Orchestration
NEUTRON
LBaaS, VPNaaS, FWaaS
What is OpenStack?

What really is OpenStack?
7
OpenStack is like the Linux Kernel

What is Rackspace Private Cloud?
8
• Rackspace Private Cloud Software is powered by OpenStack, the same cloud
platform we used to build the Rackspace public cloud. Because our Private Cloud
Software is based on open-source technology, you don't have the risk of being
locked into a proprietary platform.
RPC is the Distribution

Inaugural
Design
Summit in
Austin
Jul
OpenStack Launch!
CloudFiles
Launches
May
2008
NASA
writes Nova
Controller
First release
25+ partners
Oct
Austin
Swift prod
Nova dev preview
35+ partners
Nov
First public
Design
Summit in
San Antonio
2009
Mar
2006
Rackspace
Cloud
Launches
Source: Randy Bias & Others
History Lesson
9

2nd Summit
Jan Feb Jul
2nd release
Apr
3rd Summit
in Santa Clara
plus conference
Governance moves forward
with project technical leads
and policy board elections.
Decision to move to 6-month
release cycle over 3-month
Cactus
Nova for larger-
scale prod
Sept Oct
Diablo
Major stability release
First of 6-month releases
Rackspace
announces plans
to launch
OpenStack Foundation
Bexar
Nova for mid-sized prod
Glance added as core
Happy Birthday!
10
History Lesson

AT&T joins
OpenStack
Jan Feb May
Created
framework for
Foundation
Apr
19 companies
announce
public support
for Foundation
Drafting committee
formed - creating
legal documents
Essex
Keystone in core
Horizon in core
Aug Sep
Board
Elections
HP Cloud
launch
Oct
Framework &
documents ratified
by community
Inaugural OpenStack
Foundation Board
meeting
VMware, Intel & NEC
accepted as Gold members
Foundation
Launched!
Folsom
Cinder in core
Networking in core
11
History Lesson

Apr Oct
Grizzly
Ceilometer in incubation
HEAT in incubation
Apr
IcehouseHavana
Ceilometer in integration
HEAT in integration
LBaaS
Nov
Summit in
Hong Kong
First Summit
100% run and
funded by
Foundation
Juno
Oct
Summit in
Paris
Summit in
Atlanta
May Nov
Trove in Integration
Compute rolling upgrades
Block Storage migration
Federated Keystone
12
History Lesson

The OpenStack Programs
13
• OpenStack Compute (Nova) - integrated program since Austin release
• OpenStack Networking (Neutron) - integrated program since Folsom release
• OpenStack Object Storage (Swift) - integrated program since Austin release
• OpenStack Block Storage (Cinder) - integrated program since Folsom release
• OpenStack Identity (Keystone) - integrated program since Essex release
• OpenStack Image Service (Glance) - integrated program since Bexar release
• OpenStack Dashboard (Horizon) - integrated program since Essex release
• OpenStack Telemetry (Ceilometer) - integrated program since the Havana release
• OpenStack Orchestration (Heat) - integrated program since the Havana release
• OpenStack Database (Trove) - integrated program for Icehouse
• OpenStack Bare Metal (Ironic)
• OpenStack Queue Service (Marconi)
• OpenStack Data Processing (Hadoop) (Sahara)

Rackspace Private Cloud
Architecture

Rackspace Private Cloud Architecture
• Rackspace Private Cloud (RPC) is OpenStack
• Easy to install
• Tested configurations
• Supported
• Community (Free)
• Fanatical Support (per node)
15

• Rackspace Private Cloud (Compute) is made up of
• 2 x Controllers (HA)
• N Computes (Hypervisors)
• N Cinder nodes (Block Storage)
• Rackspace Private Cloud (Object Storage)
• 2 x Identity
• Hardware Load Balancers
• N x Proxy
• N x Storage (With DAS/JBOD)
16

17

OpenStack Architecture
18

OpenStack Architecture
19

20

Hybrid Cloud: Dedicated to Cloud
21

Hybrid Cloud: Enterprise to Cloud
22

Hybrid Cloud: Multi-Cloud
23

Intro to
Networking & SDN

Intro to Software Defined Networking
25
• OpenStack Networking: Neutron
• Networking Architecture for Rackspace Private Cloud
• Servers/Nodes
• Controller: Neutron API, Agents, Open vSwitch
• Computes: Agents and Open vSwitch
• Network Cards in each
• NIC for Host/Management
• Usually bonded into different switches for HA
• NIC for Neutron
• Usually bonded into different switches for HA
• Can be bonded for LACP
• Can have more than one NIC for different networks/speeds

Rackspace Private Cloud: Networking
26

Host or Management Network
27
• The Host Network is nothing more than the subnet the servers live on
• I.e. the OpenStack services run on this network
• Just like Apache or Bind would
• As a user of the private cloud, this is your address for the API / GUI

Provider Network
28
• This is the “Neutron” network
• Networks that go through these interfaces have been defined in OpenStack
• Neutron Provider Networks can be
• GRE Tunnels
• VLANs
• Flat Networking
• Can have multiple provider networks
• One for “Standard” traffic at 1G
• Another for “Fast” access at 10G
• When creating networks in Neutron, we can specify which NIC “bridge” to use
for this purpose

Provider Network Type: GRE
29
• GRE Tunnel networks in Neutron form a mesh in OpenStack
• Each Compute and Controller will be able to send/receive packets over these
networks
• Each GRE tunnel is given an ID

Provider Network Type: VLAN
30
• VLAN networks work as they would in a physical world
• Switch will have VLAN tags trunked on their ports
• An OpenStack user would create a Neutron network with a corresponding VLAN ID
• Providing all the switch ports have that VLAN ID, OpenStack Networking will work

Provider Network Type: Flat
31
• Flat Networking is the most basic
• It is analogous to Flat DHCP in “Nova Networking”
• A single flat structure with no network isolation

Physical Networking
32

Storage

OpenStack Block
Storage

OpenStack Block Storage
35
• OpenStack Block Storage
• Project Name Cinder
• Provides additional, usually resilient storage to instances
• Rackspace Private Cloud Supports
• Local Disk (LVM)
• EMC
• NetApp
• Solidfire (in your Datacentre)

36
• Can only attach a volume to one instance at a time
• Like a USB stick
• Typical Use Cases
• Tables for MySQL stored on Block Storage volumes
• Performance sensitive data
• Computes run SAS, use SSD-backed Block Storage
• Providing instances access to raw block storage

37
• Snapshots
• Backup and restore volumes of data
• Boot from Volume
• Ability to run image from block storage
• Good for the “Pets”
• Potentially less Orchestration
• Possibly Windows
• Volume is not deleted when an instance is terminated

What really is OpenStack?OpenStack Block Storage
38

39
• Rackspace recommends
• 1 core per 3TB capacity
• At least 6 SATA or SAS drives of at least 1TB capacity each.
• At least 2GB RAM, plus an additional 250MB RAM per TB of drive.
• RAID Controller with battery backup in RAID5 or RAID10 configuration.

Block Storage
40

OpenStack Object
Storage

Object Storage: Swift
42
• API driven Object Storage
• Upload/Download via HTTP/HTTPS
• Highly Resilient Distributed Object Storage
• Data is written multiple times (default 3)
• Rings
• Account
• Container
• Object
• Location aware: Zones
• Disk Partition
• Disk
• Server
• Cabinet
• Datacentre

43

44

45

Highly Available
Rackspace Private
Cloud

HA and Non-HA in Private Cloud
47

Designing For Failure
• Your infrastructure must be HA
• Your apps running on your cloud should tolerate failure
• Automate everything
• Automate recovery
• Use load balancers
• Use message queues
• Put workloads suitable into the cloud
• Don’t expect “Live Migration”
• Live Migration is an Operations Function, not a design choice.
48

Designing For Failure
• In Rackspace Private Cloud
• Controllers are HA
• MySQL HA (Multi-Master)
• RabbitMQ Cluster
• APIs behind HA Proxy
• Keepalived for floating IPs
• Computes
• Individual scale out units
• Local storage
• Will fail at some point
49

Highly Available RPC: Keepalived
• Keepalived
• Uses Layer4 Load Balancing Module (IPVS)
• Uses VRRP (Virtual Redundancy Router Protocol)
• Uses multicast address 224.0.0.18 by default
• Specify VRID
• Must be unique on network
• Specify the Virtual IP (Floating IP)
• Watchdog monitors the keepalived processes
• Healthcheckers monitor health of service
• VIP Fails over when check of service fail
50

Highly Available RPC: Controllers
51

Highly Available RPC: MySQL + Keepalived
• MySQL
• Running Multi-Master
• Both nodes are able to handle “writes”
• In RPC we avoid conflict by using Keepalived
• Ensure we only write to a single node
• Multi-Master allows for automatic recovery
• No manual promotion of Slave to Master
• But we treat other “Master” as a “Slave”
52

Highly Available RPC: MySQL + Keepalived
53

Highly Available RPC: RabbitMQ + Keepalived
• RabbitMQ
• Running Cluster
• Rabbit 3.x
• Use Keepalived to write to a single RabbitMQ node
• Allow failover to other clustered node
• RabbitMQ failover is complex!
54

Highly Available RPC: RabbitMQ + Keepalived
55

Highly Available RPC: API HA Proxy Keepalived
• APIs
• Nova API
• Glance API
• Keystone API
• Neutron API
• Cinder API
• Horizon
• Utilise HA Proxy
• HA Proxy configured on each controller
• Each HA Proxy config knows about the other controller too
• Hit the HA Proxy controlled by Keepalived
• But that request is load balanced across both nodes (backend)
56

Highly Available RPC: API HA Proxy Keepalived
57

Highly Available RPC: Compute
• Non-HA
• Design for failure
• Evacuate and Live Migration
• DRBD block migration
• Shared storage
• KVM: Still a pause
• Good for Operations, not for reliance on HA
58

Highly Available RPC: Block Storage
• Cinder API
• HA
• Cinder Backends
• NetApp, EMC, etc. (Enterprise Storage)
• Assumed HA/Resilient
• LVM
• Utilise local disk/DAS/JBOD
• Deploy more than one in Private Cloud
• Computes mount volume from Cinder1
• Computes also mount volume from Cinder2
• Software RAID the two volumes
59

Sandbox

RPC Sandbox
• If you are wanting hands on instance access
• http://www.rackspace.com/cloud/private
• Download Virtual Box or VMware OVA
• Also
• Vagrant (http://www.vagrantup.com/
• VirtualBox (http://www.virtualbox.org/)
• Git
• https://github.com/BigCloudSolutions/VagrantSwift
• https://github.com/OpenStackCookbook/OpenStackCookbook
61

Installation

Install Rabbit MQ
Installation of Chef
Configuration of
Chef Client
Fetch and Upload
Cookbooks
Configuration of
Environment
Bootstrapping
Nodes
Chef Client
Testing Installation
Rackspace Private
Cloud Ready

CHEF PREP INSTALL DONE
What really is OpenStack?RPC Installation Steps
63

64
Easy installation of RPC
• Head to http://www.rackspace.com/cloud/private
• Follow the instructions
• Scripts to install Chef
• Scripts to install Rackspace Private Cloud Cookbooks

65
$ mkdir -p /opt/chef-cookbooks
$ COOKBOOK_VERSION=v4.2.2
$ apt-get install git -y
$ git clone https://github.com/rcbops/chef-cookbooks.git
/opt/chef-cookbooks
$ pushd /opt/chef-cookbooks
$ git checkout ${COOKBOOK_VERSION}
$ git submodule init
$ git submodule sync
$ git submodule update
# Upload all of the RPCS Cookbooks
$ knife cookbook upload -o /opt/chef-cookbooks/cookbooks -a
$ popd
$ knife role from file /opt/chef-cookbooks/roles/*.rb
Fetch and upload Cookbooks

What really is OpenStack?Configuration of Environment
66
• Define our RPC OpenStack in a single JSON for Chef to use
• Define
• Nova Configuration (hypervisor, scheduler, etc)
• Networking
• Neutron (default type, vlans, etc)
• Glance
• Cinder
• MySQL
• Rabbit
• HA Details (VIPs to use)
• etc.

{
"name": ”rpcs",
"description": "Environment for Rackspace Private Cloud",
"cookbook_versions": {
},
"json_class": "Chef::Environment",
"chef_type": "environment",
"default_attributes": {
},
"override_attributes": {
"monitoring": {
"procmon_provider": "monit",
"metric_provider": "collectd"
},
"enable_monit": true,
"osops_networks": {
"management": "10.240.0.0/24",
"swift": "10.240.0.0/24",
"public": "10.240.0.0/24",
"nova": "10.240.0.0/24"
},
"rabbitmq": {
"cluster": true,
"erlang_cookie": "CookieContents"
},
"nova": {
"config": {
"use_single_default_gateway": false,
"ram_allocation_ratio": 1.0,
"disk_allocation_ratio": 1.0,
"cpu_allocation_ratio": 2.0,
"resume_guests_state_on_host_boot": false
},
"network": {
"provider": "neutron"
},
67

68
$ EDITOR=vi knife environment edit rpcs
Or
$ knife environment from file rpcs.json
68

$ CONTROLLER1=10.240.0.1
$ CONTROLLER2=10.240.0.2
$ COMPUTES=“10.240.0.3 10.240.0.4 10.240.0.5”
# Controllers (HA)
$ knife bootstrap –E rpcs
-r role[ha-controller1],role[single-network-node] ${CONTROLLER1}
$ knife bootstrap -E rpcs
-r role[ha-controller2],role[single-network-node] ${CONTROLLER2}
# Computes
$ for node in ${COMPUTES};do
knife bootstrap -E rpcs-r role[single-compute] ${node}
done
What really is OpenStack?Bootstrapping Nodes
69
• Bootstrapping configures and installs chef-client
• and conveniently allows us to set roles too

$ chef-client
$ ssh ${CONTROLLER2} chef-client
$ chef-client
What really is OpenStack?Make Controllers HA
70
• To make Controllers HA and utilising the VIPS and synchronise the data between
the two:
• Run chef-client on Controller1

$ knife ssh “role:single-compute” “chef-client”
What really is OpenStack?Run Chef-Client on Computes
71
• Finally, run chef-client on remaining nodes (Our Compute Nodes)

$ nova-manage service list
Or
$ . openrc
$ nova service-list
$ nova hypervisor-list
$ . openrc
$ neutron agent-list
What really is OpenStack?Testing the Installation
72
• Check Compute
• Check Networking

$ ovs-vsctl show
$ route -n
$ ssh ${CONTROLLER2} route -n
What really is OpenStack?Testing the Installation
73
• Check Open vSwitch
• Check HA / Keepalived

QUESTIONS?

RACKSPACE® HOSTING | © RACKSPACE US, INC. | RACKSPACE® AND FANATICAL SUPPORT® ARE SERVICE MARKS OF RACKSPACE US, INC. REGISTERED IN TH E UNITED STATES AND OTHER COUNTRIES.
RACKSPACE® HOSTING | 5 MILLINGTON ROAD | HAYES, UNITED KINGDOM UB3 4AZ
UK SALES: +44 (0)20 8712 6507 | UK SUPPORT: 0800 988 0300 | WWW.RACKSPACE.CO.UK

Intro to OpenStack - WAJUG

More Related Content

What's hot (20)

Similar to Intro to OpenStack - WAJUG (20)

Recently uploaded (20)

Intro to OpenStack - WAJUG