SlideShare a Scribd company logo
OpenStack in Sina




         @程辉
   freedomhui@gmail.com
Agenda
   OpenStack Overview
   Architecture Analysis
   Integration Changes
   Sina Contributions
Open stack in sina
AWS模式的巨大成功
   构建了完整的云计算生态系统
   通过Web Service(API)管理一切服务*
   完全面向服务架构SOA(Service-Oriented
    Architecture)*
   事实上的IaaS 标准
   成功的商业模式



    *https://plus.google.com/112678702228711889851/posts/eVeouesvaVX
    *http://coolshell.cn/articles/5701.html
Open stack in sina
More Detail: http://aws.amazon.com/products/
Open stack in sina
OpenStack横空出世
   目标:AWS开源实现
   Rackspace & NASA联合成立




    Swift         Nova
OpenStack Companies




      More detail: http://www.openstack.org/community/companies/
来源:《OpenStack, OpenNebula,Eucalyptus,CloudStack社区活跃度比较》http://www.qyjohn.net/?p=1856
Open Source
Apache 2.0 license, NO ‘enterprise’ version


Open Design
Open Design Summit


Open Development
Anyone can involve development process Open development management via
Launchpad & Github


Open Community
OpenStack Foundation in 2012
OpenStack Mission


   "To produce the ubiquitous Open Source
   cloud computing platform that will meet the
   needs of public and private cloud providers
   regardless of size, by being simple to
   implement and massively scalable."
OpenStack Projects

 Core Projects                         AWS       OpenStack

    OpenStack Compute(Nova)           EC2       nova

    OpenStack Object Storage(Swift)   S3        swift
    Image Service (Glance)            EBS       nova-volume
    Identity (Keystone)               ELB       Atlas-LB
    Dashboard (Horizon)               SQS       Burrow
    Network Connectivity (Quantum)    Console   Dashboard

 Community Projects
                                       IAM       Keystone
                                       VPC       Quantum
    Melange
     Altas-LB
                                       RDS       RedDwarf
 

    Crowbar
    Juju
    RedDwarf
    Burrow
Architecture Overview
Detail Overview
Where to Get Started?



                        Ubuntu 12.04 server 集成OpenStack




                           Trystack.org申请测试账号




                           devstack.sh一键安装
OpenStack Development

  Authorization                                          Hosting code &
  (group membership)        Bug tracking                 formal docs



  Feature planning     Mailing lists
  (Blueprints)
                User support
                (Answers)



             Wiki
            Informal docs


                                           Continuous
                                           integration
Nova Key Features

    ReST-base API
    Asynchronous communication
    Horizontally scalable
    Shared nothing architecture*
    Distribute everything
    Test everything
    100% Python Based

                 * http://en.wikipedia.org/wiki/Shared_nothing_architecture
                 * http://wiki.openstack.org/BasicDesignTenets
OpenStack Compute: Nova

 nova-api
 Compute API Server
 OpenStack API, EC2 compatibility API

 nova-compute
 Compute worker
 Manage compute host and VMs
 Libvirt(QEMU,KVM,LXR), XenServer and XCP, ESX(i)*

 nova-network
 Network controller
 Manage network resources: IPAM, VLAN, NAT


                                        *http://wiki.openstack.org/HypervisorSupportMatrix
OpenStack Compute: Nova(cont.)

 nova-scheduler
 Determines the placement of new resources

 nova-volume
 Block storage, remote attach a LVM volume using iSCISI protocol
 Like Amazon EBS, but far way from mature

 RabbitMQ
 Message Queue
 Cast and RPC Call for services
Keystone: Concept

 User/Tenant
 Authentication/Authorization
 Token
 Service/Endpoint
 Role
Keystone: User Case
Nova Network

 L2
 FLAT, FLATDHCP, VLAN
 L3
 IPAM(IP Address Management)
 Fixed IP, Floating IP
 Gateway, NAT, VPN
Quantum

Quantum Basics
Nova: virtual server
Quantum: virtual network
Quantum :
   Expose a API for creating virtual networks and attaching
    instances(e.g.,novaservers) to those networks
   Manage switches(virtual or physical) in the data center to
    implement connectivity described via API
   Provide a“plugin” architecture to leverage support using different
    back-end technologies
Open stack in sina
Quantum: available plugins
 Open vSwitch
 - Builds isolated networks with OVS and L2-in-L3 tunnel
 Cisco UCS
 - Isolation based on VLAN and net-profiles applied to Cisco
 UCS converged network adapters
 Linux Bridge
  - Build isolated networks with VLAN interfaces and linux
 bridges
  - Works with every Linux Distro
 NTT-Data Ryu
  - Acts as a proxy for the NTT Ryu platform
 Nicira NVP
  - Acts as a proxy for the Nicira NVP platform
Swift: Storage Types


    Types              Protocol              Application
    Block Storage      SATA, SCISI, iSCISI   SAN, NAS, EBS
    File Storage       Ext3/4, XFS, NTFS     PC, Servers, NFS
    Object Storage     HTTP, REST            Amazon S3,
                                             Google Cloud Storage,
                                             Rackspace Cloud Files
    Specific Storage   Specific protocol     MySQL, MongoDB,
                       based on tcp          HDFS




   We want a Object Storage like Amazon S3.
Features                     Swift       Amazon S3
            object/bucket CRUD                   √        √
            account/bucket/object ACL            √        √
            object metadate                      √        √
            large object                         √        √
            rate limit                           √        √
  Swift     expiring object                      √        √

   vs       static web                           √        √
            REST API                             √        √
Amazon S3   Account support                      √        X
            Account metadata                     √        X
            Bucket metadata                      √        X
            Bucket sync across cluster           √        X
            Object versioning                    X        √
            Log to bucket                        X        √
            Notification                         X        √
            Reduced Redundancy Storage           X        √
            SOAP API                             X        √
            Server Side Encryption               X        √
            BitTorrent protocol                  X        √
Swift Evaluation

     Extremely Durable and Highly Available

     Superior Scalability

     Linear Growth of Performance

     Symmetric Architecture

     No Single-failure

     Simple & Reliable
Swift Components

    The Ring: Mapping of names to entities (accounts,containers,
     objects) on disk.
           Stores data based on zones, devices, partitions, and replicas
           Weights can be used to balance the distribution of partitions
           Used by the Proxy Server for many background processes
    Proxy Server: Request routing, exposes the public API
    Replication: Keep the system consistent, handle failures
    Updaters: Process failed or queued updates
    Auditors: Verify integrity of objects, containers, and account
Swift Architecture
                                                     • 1 Zone = 1 Physical Server with 12x2T disk
                          GET abc.png                • Write/Read applies quorum protocol
                  PUT abc.png


                                   Load Balancer


    Zone1            Zone2                Zone3                  Zone4                 Zone5

 Proxy Server     Proxy Server         Proxy Server           Proxy Server          Proxy Server




 Object Server    Object Server        Object Server         Object Server          Object Server

   Container        Container            Container              Container             Container
    Server           Server               Server                 Server                Server

 Account Server   Account Server       Account Server         Account Server        Account Server
                                                                               31
Swift Installation

  Swift packages
   Proxy Server
  Account Server
  Container Server
   Object Server             Physical Deployment




                                         Storage Nodes
 OS installation


      sda            sdb          sdc        sdd               sdk
      raid 1                                             ……
disk1      disk2     disk3       disk4      disk5             disk12
Conclusion
    核心功能基本可用,但稳定性需要加强
    云服务(web service)比较丰富
    起步虽晚,但发展飞快,OpenStack生态系统正
     在形成
    逻辑结构清晰、文档丰富、源码规范易懂,便
     于二次开发
    Open [Source | Desgin | Development |
     Community]
Integration Challenges
   Best Network Topology
   Security Enhancement
   Load Balancer
   CDN Services
   Metering & Billing
Infrastructure & Platform
Physical Servers
Traditional Operation

Virtualization Platform(IaaS)
●VM Management System(VMMS) → Sina Web
Service(SWS)
●VMMS is private solution developed in-house

●SWS is based on OpenStack


                                                 新浪云计算
Application Platform(PaaS)
●Virtual Host → Sina App Engine(SAE)
●SAE provides both Public and Private Service.

●Proved to be Efficient and Robust
Nova Network
Networking is the biggest challenges for IaaS
Network Topology:
•   VLAN
•   FlatDHCP
•   FlatDHCP & Multihost
Network Topology (VLAN)
Capability:
• Accessibility of VMs within one tenant
• Isolation of VMs from different tenants
• VM is able to access public network
• VM can be accessible from public network
• Isolation between virtual network and
  internal network




Drawback:
• Pre-allocate network for future projects
• Hard-limit of vlan 4096
• Traffic bottleneck in the gateway/NAT
Network Topology(Flat)
Capability:
• Accessibility of all VMs in the fixed IP
  range
• VM is able to access public network
• VM can be accessible from public network
• Full isolation between virtual network and
  internal network

Bonus:
• Do not need pre-allocate for new projects
• Eliminating bottleneck between tenants

Drawback:
• Tenant isolation has gone
• Traffic bottleneck still exists in NAT
Network Topology(Flat & Multihost)
Capability:
• Accessibility of all VMs in the fixed IP range
• VM is able to access public network
• VM can be accessible from public network

Bonus:
• Totally distributed architecture avoid single-
  point failure.
• Multiple gateway eliminates NAT bottleneck
• High speed between OS regions

Drawback:
• Tenant isolation lessens
• Need security facility(SWS-filter) to protect
  intranet




   If security problems were solved, this would be our   best choice!
Security in OpenStack
Security Group --- L3 Filter               Static filters --- L2 Filter
Role-based firewall                        MAC, IP, and ARP spoofing protection
  One security group is a Role             Not configurable
Ingress filtering                           Defined in /etc/libvirt/nwfilter/*.xml
  Target is the instance                  Implemented by ebtables
  Source can be CIDR or another group      ebtables -t nat --list
Implemented by iptables
  See details: iptables -t filter -n -L
  Whitelist mechanism(ACCEPT rules)
Security Enhancement

SWS Filter
Prevent Intranet Penetration
• Intranet is the internal network outside of
  OpenStack
Egress filtering
• Target is internal network
• Source is instances in OpenStack
Implementation
• Whitelist mechanism(ACCEPT rules)
• On the top of nova-filter-top Forward
  Chain
Rational
• SWS filter is managed by cloud manager
• Only explicit authorized packets can reach Internal network C
• Packet should be controlled within Compute Node
Security Enhancement
Security Group VS SWS Filter
Load Balancer
 Goals
 Load Balance
 • Dispatch request                                  DNS Acceleration Design
 • Support multiple routing algorithm
 • Health check
                                                                         Smart DNS
 Acceleration
 • Reality: narrow bandwidth between ISPs
 • Building fiber channels from ISPs to pivot        Public Network
 • Given the same endpoint within user’s ISP
                                           Telecom     Unicom      Mobile        Others ISP
 IPv4 Shortage
 • Reality: dozens of public IPs support
    hundreds of VMs                                   High speed fiber channel
 • IPv4 has been exhausted
 • IPv6 is not realistic yet in China
                                                                 Pivot
L7 Load Balancer
 Layer 7 Load Balancer

 Consideration:
 1. dispatch request by Host header
 2. nginx module
L4 Load Balancer
  Layer 4 Load Balancer

  Consideration:
  1. dispatch request by TCP port
  2. lvs + haproxy




ssh –p 2000 root@socket.abc.com
Kanyun: Monitoring system
 Compute
                 Worker
  Network                            RDBMS                               Dashboard

  Storage

Worker
                Retrieve
                 usage
                  info


                                   API daemon                              Billing
 Aggregator
                                Responds to client
            Calculates/stores       request
                metrics


                                 http://github.com/lzyeval/kanyun   46
  NoSQL
RPC
Dough:Billing system                                           Database
                                                                Client

 Compute

  Network                   RDBMS                             Dashboard

  Storage




                Collector


 Monitoring
                                         Farmer              API daemon
 (Metering)

                                       Dispatch jobs          Subscribe or
                Collector                                     unsubscribe
                                                               products /
                                                               Query info
               Check status /
              Retrieve usage /                            47
                                 http://github.com/lzyeval/dough
              Create purchases
Sina Contributions

            Top 10 Bugfix Company




                                    48
Sina Contributions

   Sina creating open source project “Dough” to contribute
    metering & billing capability
   Present in OpenStack Design Summit & Conference




                                                              49
Q&A

@程辉
Technical Manager @ Sina

Twitter:@freedomhui
Email:chenghui@staff.sina.com.cn
Gtalk: freedomhui@gmail.com

More Related Content

KEY
OpenStack Report
PDF
iPaas with Fuse Fabric Technology
PPTX
Cassandra Performance and Scalability on AWS
PDF
Mirantis Folsom Meetup Intro
ODP
Supporting and Using EC2/CIMI on top of Cloud Environments via Deltacloud
PDF
Storage 101: Rook and Ceph - Open Infrastructure Denver 2019
PDF
Geek Week 2016 - Deep Dive To Openstack
PPT
Apache CloudStack AlpesJUG
OpenStack Report
iPaas with Fuse Fabric Technology
Cassandra Performance and Scalability on AWS
Mirantis Folsom Meetup Intro
Supporting and Using EC2/CIMI on top of Cloud Environments via Deltacloud
Storage 101: Rook and Ceph - Open Infrastructure Denver 2019
Geek Week 2016 - Deep Dive To Openstack
Apache CloudStack AlpesJUG

What's hot (20)

PPT
CloudStack for Java User Group
PDF
Ceph storage for ocp deploying and managing ceph on top of open shift conta...
PDF
Openstack Global Meetup
PDF
The container revolution, and what it means to operators.pptx
PPT
DevCloud and CloudMonkey
PPT
Intro to CloudStack Build a Cloud Day
PPTX
New Ceph capabilities and Reference Architectures
PDF
Ceph as software define storage
PPTX
實際架構實踐演化與解決方案
PDF
Intro into Rook and Ceph on Kubernetes
PPTX
What you need to know about ceph
PDF
Kubernetes Day 2017 - Build, Ship and Run Your APP, Production !!
PPTX
QCT Ceph Solution - Design Consideration and Reference Architecture
PPTX
Ceph and Openstack in a Nutshell
PDF
Storage tiering and erasure coding in Ceph (SCaLE13x)
PDF
Red Hat Storage 2014 - Product(s) Overview
ZIP
Constructing Web APIs with Rack, Sinatra and MongoDB
PPTX
Terraform Modules Restructured
PDF
OpenStack in Enterprise
PDF
[OpenStack Days Korea 2016] Track4 - Deep Drive: k8s with Docker
CloudStack for Java User Group
Ceph storage for ocp deploying and managing ceph on top of open shift conta...
Openstack Global Meetup
The container revolution, and what it means to operators.pptx
DevCloud and CloudMonkey
Intro to CloudStack Build a Cloud Day
New Ceph capabilities and Reference Architectures
Ceph as software define storage
實際架構實踐演化與解決方案
Intro into Rook and Ceph on Kubernetes
What you need to know about ceph
Kubernetes Day 2017 - Build, Ship and Run Your APP, Production !!
QCT Ceph Solution - Design Consideration and Reference Architecture
Ceph and Openstack in a Nutshell
Storage tiering and erasure coding in Ceph (SCaLE13x)
Red Hat Storage 2014 - Product(s) Overview
Constructing Web APIs with Rack, Sinatra and MongoDB
Terraform Modules Restructured
OpenStack in Enterprise
[OpenStack Days Korea 2016] Track4 - Deep Drive: k8s with Docker
Ad

Similar to Open stack in sina (20)

PPT
Swift Architecture and Practice, by Alex Yang
PPT
Open stackapac swift_alexyang
PDF
Open Stack Cheng Du Swift Alex Yang
PDF
OpenStack at Xen summit Asia
PPT
Ram chinta hug-20120922-v1
PDF
What is OpenStack and the added value of IBM solutions
PDF
OpenStack Tutorial
PDF
CloudStack-Developer-Day
PDF
OpenStack for VMware Administrators
PDF
CloudStack Architecture Future
PPTX
Web Scale Applications using NeflixOSS Cloud Platform
PDF
OpenCms Days 2012 - OpenCms on open clouds
PDF
WSO2 Carbon and WSO2 Stratos Summer Release Roundup
PDF
AWS reinvent 2019 recap - Riyadh - Containers and Serverless - Paul Maddox
PDF
KT ucloud storage, by Jaesuk Ahn
PDF
Am 02 osac_kt_swift
PPTX
An Intrudction to OpenStack 2017
PDF
Scalable Architecture on Amazon AWS Cloud - Indicthreads cloud computing conf...
PDF
Visão Técnica - RHOS (Red Hat Enterprise Linux OpenStack)
PDF
Cisco: Cassandra adoption on Cisco UCS & OpenStack
Swift Architecture and Practice, by Alex Yang
Open stackapac swift_alexyang
Open Stack Cheng Du Swift Alex Yang
OpenStack at Xen summit Asia
Ram chinta hug-20120922-v1
What is OpenStack and the added value of IBM solutions
OpenStack Tutorial
CloudStack-Developer-Day
OpenStack for VMware Administrators
CloudStack Architecture Future
Web Scale Applications using NeflixOSS Cloud Platform
OpenCms Days 2012 - OpenCms on open clouds
WSO2 Carbon and WSO2 Stratos Summer Release Roundup
AWS reinvent 2019 recap - Riyadh - Containers and Serverless - Paul Maddox
KT ucloud storage, by Jaesuk Ahn
Am 02 osac_kt_swift
An Intrudction to OpenStack 2017
Scalable Architecture on Amazon AWS Cloud - Indicthreads cloud computing conf...
Visão Técnica - RHOS (Red Hat Enterprise Linux OpenStack)
Cisco: Cassandra adoption on Cisco UCS & OpenStack
Ad

More from Hui Cheng (20)

PDF
OpenStack in China
PPT
Island: Local Storage Volume for Cinder
PDF
DevOps in a Public OpenStack Cloud - Hui Cheng
PPTX
OpenStack Foundation Updates
PPTX
COSUG updates
PPT
Cinder intro@open stack china-tour-shenzhen
PPTX
Lessons from Building OpenStack Public Cloud
PDF
Open stack with_openflowsdn-torii
PPTX
Open Source Software & Open Source Hardware
PPTX
OpenStack Network Planning
PPT
Asia pacopenstack joe-draft 2012-08-08
PPT
The Application of OpenStack in 360buy, by Bin He
PPT
Distributed Block-level Storage Management for OpenStack, by Danile lee
PPTX
Cloud Based VDI with OpenStack, by Shifen Yang
PPTX
Openstorage with OpenStack, by Bradley
PPTX
MAKING MONEY from openstack
PPTX
Dell OpenStack Powered Cloud Solution and Case Sharing
PPTX
Network Service in OpenStack Cloud, by Yaohui Jin
PDF
China OpenStack Community Status
PPTX
OpenStack Chances and Chanlleges_OSAC_by_ChengHui
OpenStack in China
Island: Local Storage Volume for Cinder
DevOps in a Public OpenStack Cloud - Hui Cheng
OpenStack Foundation Updates
COSUG updates
Cinder intro@open stack china-tour-shenzhen
Lessons from Building OpenStack Public Cloud
Open stack with_openflowsdn-torii
Open Source Software & Open Source Hardware
OpenStack Network Planning
Asia pacopenstack joe-draft 2012-08-08
The Application of OpenStack in 360buy, by Bin He
Distributed Block-level Storage Management for OpenStack, by Danile lee
Cloud Based VDI with OpenStack, by Shifen Yang
Openstorage with OpenStack, by Bradley
MAKING MONEY from openstack
Dell OpenStack Powered Cloud Solution and Case Sharing
Network Service in OpenStack Cloud, by Yaohui Jin
China OpenStack Community Status
OpenStack Chances and Chanlleges_OSAC_by_ChengHui

Recently uploaded (20)

PPT
“AI and Expert System Decision Support & Business Intelligence Systems”
PDF
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
PDF
KodekX | Application Modernization Development
PDF
Chapter 3 Spatial Domain Image Processing.pdf
PDF
Modernizing your data center with Dell and AMD
PPTX
20250228 LYD VKU AI Blended-Learning.pptx
PDF
[발표본] 너의 과제는 클라우드에 있어_KTDS_김동현_20250524.pdf
PPTX
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
PPTX
PA Analog/Digital System: The Backbone of Modern Surveillance and Communication
PDF
Empathic Computing: Creating Shared Understanding
PDF
Advanced IT Governance
PDF
Shreyas Phanse Resume: Experienced Backend Engineer | Java • Spring Boot • Ka...
PPTX
Cloud computing and distributed systems.
PDF
cuic standard and advanced reporting.pdf
PPTX
MYSQL Presentation for SQL database connectivity
PDF
Optimiser vos workloads AI/ML sur Amazon EC2 et AWS Graviton
PDF
Reach Out and Touch Someone: Haptics and Empathic Computing
PDF
Network Security Unit 5.pdf for BCA BBA.
PDF
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
PPTX
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
“AI and Expert System Decision Support & Business Intelligence Systems”
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
KodekX | Application Modernization Development
Chapter 3 Spatial Domain Image Processing.pdf
Modernizing your data center with Dell and AMD
20250228 LYD VKU AI Blended-Learning.pptx
[발표본] 너의 과제는 클라우드에 있어_KTDS_김동현_20250524.pdf
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
PA Analog/Digital System: The Backbone of Modern Surveillance and Communication
Empathic Computing: Creating Shared Understanding
Advanced IT Governance
Shreyas Phanse Resume: Experienced Backend Engineer | Java • Spring Boot • Ka...
Cloud computing and distributed systems.
cuic standard and advanced reporting.pdf
MYSQL Presentation for SQL database connectivity
Optimiser vos workloads AI/ML sur Amazon EC2 et AWS Graviton
Reach Out and Touch Someone: Haptics and Empathic Computing
Network Security Unit 5.pdf for BCA BBA.
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy

Open stack in sina

  • 1. OpenStack in Sina @程辉 freedomhui@gmail.com
  • 2. Agenda  OpenStack Overview  Architecture Analysis  Integration Changes  Sina Contributions
  • 4. AWS模式的巨大成功  构建了完整的云计算生态系统  通过Web Service(API)管理一切服务*  完全面向服务架构SOA(Service-Oriented Architecture)*  事实上的IaaS 标准  成功的商业模式 *https://plus.google.com/112678702228711889851/posts/eVeouesvaVX *http://coolshell.cn/articles/5701.html
  • 8. OpenStack横空出世  目标:AWS开源实现  Rackspace & NASA联合成立 Swift Nova
  • 9. OpenStack Companies More detail: http://www.openstack.org/community/companies/
  • 11. Open Source Apache 2.0 license, NO ‘enterprise’ version Open Design Open Design Summit Open Development Anyone can involve development process Open development management via Launchpad & Github Open Community OpenStack Foundation in 2012
  • 12. OpenStack Mission "To produce the ubiquitous Open Source cloud computing platform that will meet the needs of public and private cloud providers regardless of size, by being simple to implement and massively scalable."
  • 13. OpenStack Projects Core Projects AWS OpenStack  OpenStack Compute(Nova) EC2 nova  OpenStack Object Storage(Swift) S3 swift  Image Service (Glance) EBS nova-volume  Identity (Keystone) ELB Atlas-LB  Dashboard (Horizon) SQS Burrow  Network Connectivity (Quantum) Console Dashboard Community Projects IAM Keystone VPC Quantum  Melange Altas-LB RDS RedDwarf   Crowbar  Juju  RedDwarf  Burrow
  • 16. Where to Get Started? Ubuntu 12.04 server 集成OpenStack Trystack.org申请测试账号 devstack.sh一键安装
  • 17. OpenStack Development Authorization Hosting code & (group membership) Bug tracking formal docs Feature planning Mailing lists (Blueprints) User support (Answers) Wiki Informal docs Continuous integration
  • 18. Nova Key Features  ReST-base API  Asynchronous communication  Horizontally scalable  Shared nothing architecture*  Distribute everything  Test everything  100% Python Based * http://en.wikipedia.org/wiki/Shared_nothing_architecture * http://wiki.openstack.org/BasicDesignTenets
  • 19. OpenStack Compute: Nova nova-api Compute API Server OpenStack API, EC2 compatibility API nova-compute Compute worker Manage compute host and VMs Libvirt(QEMU,KVM,LXR), XenServer and XCP, ESX(i)* nova-network Network controller Manage network resources: IPAM, VLAN, NAT *http://wiki.openstack.org/HypervisorSupportMatrix
  • 20. OpenStack Compute: Nova(cont.) nova-scheduler Determines the placement of new resources nova-volume Block storage, remote attach a LVM volume using iSCISI protocol Like Amazon EBS, but far way from mature RabbitMQ Message Queue Cast and RPC Call for services
  • 21. Keystone: Concept User/Tenant Authentication/Authorization Token Service/Endpoint Role
  • 23. Nova Network L2 FLAT, FLATDHCP, VLAN L3 IPAM(IP Address Management) Fixed IP, Floating IP Gateway, NAT, VPN
  • 24. Quantum Quantum Basics Nova: virtual server Quantum: virtual network Quantum :  Expose a API for creating virtual networks and attaching instances(e.g.,novaservers) to those networks  Manage switches(virtual or physical) in the data center to implement connectivity described via API  Provide a“plugin” architecture to leverage support using different back-end technologies
  • 26. Quantum: available plugins Open vSwitch - Builds isolated networks with OVS and L2-in-L3 tunnel Cisco UCS - Isolation based on VLAN and net-profiles applied to Cisco UCS converged network adapters Linux Bridge - Build isolated networks with VLAN interfaces and linux bridges - Works with every Linux Distro NTT-Data Ryu - Acts as a proxy for the NTT Ryu platform Nicira NVP - Acts as a proxy for the Nicira NVP platform
  • 27. Swift: Storage Types Types Protocol Application Block Storage SATA, SCISI, iSCISI SAN, NAS, EBS File Storage Ext3/4, XFS, NTFS PC, Servers, NFS Object Storage HTTP, REST Amazon S3, Google Cloud Storage, Rackspace Cloud Files Specific Storage Specific protocol MySQL, MongoDB, based on tcp HDFS We want a Object Storage like Amazon S3.
  • 28. Features Swift Amazon S3 object/bucket CRUD √ √ account/bucket/object ACL √ √ object metadate √ √ large object √ √ rate limit √ √ Swift expiring object √ √ vs static web √ √ REST API √ √ Amazon S3 Account support √ X Account metadata √ X Bucket metadata √ X Bucket sync across cluster √ X Object versioning X √ Log to bucket X √ Notification X √ Reduced Redundancy Storage X √ SOAP API X √ Server Side Encryption X √ BitTorrent protocol X √
  • 29. Swift Evaluation  Extremely Durable and Highly Available  Superior Scalability  Linear Growth of Performance  Symmetric Architecture  No Single-failure  Simple & Reliable
  • 30. Swift Components  The Ring: Mapping of names to entities (accounts,containers, objects) on disk.  Stores data based on zones, devices, partitions, and replicas  Weights can be used to balance the distribution of partitions  Used by the Proxy Server for many background processes  Proxy Server: Request routing, exposes the public API  Replication: Keep the system consistent, handle failures  Updaters: Process failed or queued updates  Auditors: Verify integrity of objects, containers, and account
  • 31. Swift Architecture • 1 Zone = 1 Physical Server with 12x2T disk GET abc.png • Write/Read applies quorum protocol PUT abc.png Load Balancer Zone1 Zone2 Zone3 Zone4 Zone5 Proxy Server Proxy Server Proxy Server Proxy Server Proxy Server Object Server Object Server Object Server Object Server Object Server Container Container Container Container Container Server Server Server Server Server Account Server Account Server Account Server Account Server Account Server 31
  • 32. Swift Installation Swift packages Proxy Server Account Server Container Server Object Server Physical Deployment Storage Nodes OS installation sda sdb sdc sdd sdk raid 1 …… disk1 disk2 disk3 disk4 disk5 disk12
  • 33. Conclusion  核心功能基本可用,但稳定性需要加强  云服务(web service)比较丰富  起步虽晚,但发展飞快,OpenStack生态系统正 在形成  逻辑结构清晰、文档丰富、源码规范易懂,便 于二次开发  Open [Source | Desgin | Development | Community]
  • 34. Integration Challenges  Best Network Topology  Security Enhancement  Load Balancer  CDN Services  Metering & Billing
  • 35. Infrastructure & Platform Physical Servers Traditional Operation Virtualization Platform(IaaS) ●VM Management System(VMMS) → Sina Web Service(SWS) ●VMMS is private solution developed in-house ●SWS is based on OpenStack 新浪云计算 Application Platform(PaaS) ●Virtual Host → Sina App Engine(SAE) ●SAE provides both Public and Private Service. ●Proved to be Efficient and Robust
  • 36. Nova Network Networking is the biggest challenges for IaaS Network Topology: • VLAN • FlatDHCP • FlatDHCP & Multihost
  • 37. Network Topology (VLAN) Capability: • Accessibility of VMs within one tenant • Isolation of VMs from different tenants • VM is able to access public network • VM can be accessible from public network • Isolation between virtual network and internal network Drawback: • Pre-allocate network for future projects • Hard-limit of vlan 4096 • Traffic bottleneck in the gateway/NAT
  • 38. Network Topology(Flat) Capability: • Accessibility of all VMs in the fixed IP range • VM is able to access public network • VM can be accessible from public network • Full isolation between virtual network and internal network Bonus: • Do not need pre-allocate for new projects • Eliminating bottleneck between tenants Drawback: • Tenant isolation has gone • Traffic bottleneck still exists in NAT
  • 39. Network Topology(Flat & Multihost) Capability: • Accessibility of all VMs in the fixed IP range • VM is able to access public network • VM can be accessible from public network Bonus: • Totally distributed architecture avoid single- point failure. • Multiple gateway eliminates NAT bottleneck • High speed between OS regions Drawback: • Tenant isolation lessens • Need security facility(SWS-filter) to protect intranet If security problems were solved, this would be our best choice!
  • 40. Security in OpenStack Security Group --- L3 Filter Static filters --- L2 Filter Role-based firewall MAC, IP, and ARP spoofing protection  One security group is a Role  Not configurable Ingress filtering  Defined in /etc/libvirt/nwfilter/*.xml  Target is the instance Implemented by ebtables  Source can be CIDR or another group  ebtables -t nat --list Implemented by iptables  See details: iptables -t filter -n -L  Whitelist mechanism(ACCEPT rules)
  • 41. Security Enhancement SWS Filter Prevent Intranet Penetration • Intranet is the internal network outside of OpenStack Egress filtering • Target is internal network • Source is instances in OpenStack Implementation • Whitelist mechanism(ACCEPT rules) • On the top of nova-filter-top Forward Chain Rational • SWS filter is managed by cloud manager • Only explicit authorized packets can reach Internal network C • Packet should be controlled within Compute Node
  • 43. Load Balancer Goals Load Balance • Dispatch request DNS Acceleration Design • Support multiple routing algorithm • Health check Smart DNS Acceleration • Reality: narrow bandwidth between ISPs • Building fiber channels from ISPs to pivot Public Network • Given the same endpoint within user’s ISP Telecom Unicom Mobile Others ISP IPv4 Shortage • Reality: dozens of public IPs support hundreds of VMs High speed fiber channel • IPv4 has been exhausted • IPv6 is not realistic yet in China Pivot
  • 44. L7 Load Balancer Layer 7 Load Balancer Consideration: 1. dispatch request by Host header 2. nginx module
  • 45. L4 Load Balancer Layer 4 Load Balancer Consideration: 1. dispatch request by TCP port 2. lvs + haproxy ssh –p 2000 root@socket.abc.com
  • 46. Kanyun: Monitoring system Compute Worker Network RDBMS Dashboard Storage Worker Retrieve usage info API daemon Billing Aggregator Responds to client Calculates/stores request metrics http://github.com/lzyeval/kanyun 46 NoSQL
  • 47. RPC Dough:Billing system Database Client Compute Network RDBMS Dashboard Storage Collector Monitoring Farmer API daemon (Metering) Dispatch jobs Subscribe or Collector unsubscribe products / Query info Check status / Retrieve usage / 47 http://github.com/lzyeval/dough Create purchases
  • 48. Sina Contributions Top 10 Bugfix Company 48
  • 49. Sina Contributions  Sina creating open source project “Dough” to contribute metering & billing capability  Present in OpenStack Design Summit & Conference 49
  • 50. Q&A @程辉 Technical Manager @ Sina Twitter:@freedomhui Email:chenghui@staff.sina.com.cn Gtalk: freedomhui@gmail.com