云计算平台存储架构设计@邓海韬Nicko
- 2. Agenda • 介绍 – 基础架构,四要素,内容和相关文档 • Availability(可靠性) • Secure Separation(安全隔离) • Service Assurance(服务保证) • Management (可管理性) • Storage in SMT Architecture(SMT架构中的存储系统) • Summary(总结)
- 3. Secure Multi-Tenancy架构 传统数据中心架构 Secure Multi-tenancy 安全多租赁架构 Core Apps HR Apps BU Apps VMware VMware VMware HR BU APP HR BU APP
- 4. Secure Multi-Tenancy架构四要素 可用性 安全隔离 服务保证 s 可管理性 • 构建弹性架构, • 实现跨分区的数 • 提供跨主机层、 • 简化管理 提供高可用性、 据安全隔离 网络层和存储层 • 端到端的管理 容错性和冗余 的一致性的SLA 性 • 增加安全和访问 保证 控制 Secure Multi-tenancy Architecture
- 5. Secure Multi-Tenancy基础架构构成 Compute VMware vShield 服务器 Manager VMware vShield VMware vShield VMware vSphere VMware vCenter Cisco Unified Cisco Nexus 1000V Computing System NetApp VMware vSphere VMware vSphere VMware vSphere SnapManager 网络 Cisco UCS 5100 Cisco Nexus 1000V Blade Server Cisco Nexus 5000 Cisco UCS Cisco Nexus 7000 Cisco UCS 6100 Manager Cisco MDS Fabric Interconnect NetApp SANscreen 存储 Network NetApp Unify FAS Storage Cisco Nexus 5000 NetApp Multistore Flexvol/De-Duplication Cisco Data Center Flexshare Cisco Nexus 7000 Network Manager Snapshot/Snapmirror 管理 SAN VMware vShield Manager Cisco MDS VMware vCenter Cisco UCS Manager Cisco DC Network Manager Storage NetApp Operations Manager NetApp MultiStore NetApp FilerView NetApp Provisioning Manager NetApp Provisioning Manager NetApp Protection Manager NetApp FAS NetApp Operations Manager NetApp SANscreen & SnapManager
- 6. Secure Multi-Tenancy基础架构数据访问示意图 Computer Network Storage 7
- 7. 架构设计指南 Solution Brief (4 pages) Architecture Overview (25 pages) Designing Secure Multi-tenancy into Virtualized Data Center CVD: Design Guide Design Guide (90 pages) Cisco Validated Design (CVD) Design Considerations Best Practice Bill of Material CVD: Deployment Guide (100+ pages) Configuration Software Recommendation
- 8. Agenda • 介绍 – 基础架构,四要素,内容和相关文档 • Availability(可靠性) • Secure Separation(安全隔离) • Service Assurance(服务保证) • Management (可管理性) • Storage in SMT Architecture(SMT架构中的存储系统) • Summary(总结)
- 9. 极富弹性的端到端基础架构 Core/ Aggregation Cisco 服务器 Nexus 7000 vCenter Heartbeat VMware HA Access vPC Cisco vMotion/Storage vMotion Nexus 5000 UCS Fabric Redundancy Compute vPC vPC Cisco UCS 6100 Fabric 网络 Interconnect Nexus 1000V vPC VMware 4x10GE 4x10GE EtherChannel vCenter UCS 5100 VMware vSphere 4x10GE 4x10GE Blade Server N1KV Active/Standby VSM Link/Device Redundancy SAN/Storage Cisco MDS FC FC 存储 Ether Channel FC FC Ether Channel RAID-DP NetApp HA NetApp FAS 10GE 10GE Snapshot SnapMirror/SnapVault
- 10. 基础架构可靠性:网络和UCS Nexus 7000 and Nexus 5000 Aggregation Layer Loopless Topology with vPC Port-Channel RPVST+ vPC Access Layer Unified Computing System Fabric Availability Control Plane Availability Forwarding Path Availability Blade Server Path Availability vPC vPC vPC vPC Nexus 1000V A UCS 1 B A UCS 2 B Nexus 1000V x4 x4 x4 x4 Supervisor Availability VSM Active (VSM) x4 x4 x4 x4 VEM Forwarding Path VSM Standby Availability (VEM) x4 x4 x4 x4 x4 x4 x4 x4
- 11. 基础架构可靠性:VMware HA和vCenter心跳 • VMware HA – Protection against server failure • Configurable VM restart priority – Protection against VM guest OS failure • Configurable VM heartbeat monitor sensitivity – Primary vs. Secondary Nodes • vCenter Heartbeat – Primary and Secondary vCenter server in replication and synchronization – Protection against hardware and application failure
- 12. 基础架构可靠性:vMotion和Storage vMotion • vMotion – Continuously availability to tenants during planned server outages • Zero downtime migration of VM between servers • Storage vMotion – Continuously availability to tenants during migration to different tiers of storage • Supports all three protocols: NFS, iSCSI, FCP
- 13. 基础架构可靠性:基于存储系统的全面数据保护手段 SnapManager Suit 与应用高度集成 的数据管理套件 Synchronous Clusters App 不中断的操作 Integration MetroCluster 完整的数据恢复方案 Continuous Synchronous SnapMirror® 基于网络的恢复 Availability Local Snapshots Asynchronous SnapMirror 不同级别的服务协议 Backup SnapVault®, SnapLock®, LockVault™ 磁盘备份及数据 遵循方案 DR/Dev/Test SnapRestore® 数据快速恢复 Clones Snapshot™ Copies – D2D Backup 数据快速备份和恢复 Disaster Recovery Compliance 14 NetApp Confidential - Limited Use
- 14. Agenda • 介绍 – 基础架构,四要素,内容和相关文档 • Availability(可靠性) • Secure Separation(安全隔离) • Service Assurance(服务保证) • Management (可管理性) • Storage in SMT Architecture(SMT架构中的存储系统) • Summary(总结)
- 15. 数据安全隔离 服务器 网络 存储 UCS & vSphere RBAC Access Control List vFiler units VM Security with VLAN Segmentation IP Spaces vShield and Nexus 1000V QoS - Classification VLAN Segmentation UCS Resource Pool Separation
- 16. 访问控制 Cloud Administrator 定义角色 云架构管理员 租赁单元管理员 NetApp MultiStore 租赁单元用户 vFiler vFiler vFiler vFiler 基于角色的访问控制 Tenant A Tenant B Tenant C Tenant D UCS Manager Tenant B Server Admin Network Admin Storage Admin Customized Admin vCenter Privilege Assignment User Group Association Permission Assignment 访问控制列表 Nexus 1000V, 5000, 7000
- 17. VLAN整合 VLAN Types Function Routable Control Plane VLAN To Mange control Plane No Management VLAN To Mange Management Yes Engineering VLAN Marketing VLAN To service Marketing team. 复杂 To separate for Engineering. No Depends HR VLAN To to service HR group. No Data Center VLAN To separate Data Center from other places. Depends Storage VLAN VLAN only for SAN No Perspective VLAN组 功能 Routable Cloud Admin Management Manage UCS, N1KV, Yes Storage, Network Devices Network Management & OOB Control Connectivity across compute, network, and No storage entities Tenant(租赁单元) Application Admin VM and application administration Yes Data Service the customer application. Depends
- 18. VM安全和vShield和Nexus 1000V 与N1KV完全集成 •Virtual Service Domain (VSD) feature Tenant A Tenant B Tenant C leveraged by vShield to intercept VM- destined flows 安全隔离 •Simple container-based rule creation leveraging vCenter inventory objects •Point of enforcement close to VM •Policy based separation between tenants •Policy based separation for multi-tier application Protected Members of VSD (VSD Inside) Nexus 1000V vMotion awareness • vShield session state tables follow the Unprotected VM (VSD Outside) •Cisco VN-Link maintains VM protection policy consistency during vMotion Physical Adapters 19
- 19. 计算资源的隔离 vSphere Resource Pool Design Best Practice Dedicated resource pools for infrastructure and tenants Separate sub-resource pool for individual tenants Combined with RBAC to securely isolate access between tenants Tenant B Resource Pool Storage Pool Interconnect Pool Tenant A Tenant B Tenant B Resource Pool Resource Pool Resource Pool Tenant Resource Pool Infrastructure Resource Pool
- 20. NetApp Secure Multi-Tenancy 用户/应用/符合的分区隔离 Challenges Resource utilization MultiStore® Secure separation Resource hogs Customer A Customer B Customer C Data Data Data Secure multi-tenancy MultiStore Secure partition of storage and Data Data Data networking Data Data Data Proven technology: 16,000 licenses Virtual Storage Virtual Storage Virtual Storage Third-party valid security testing Partition Partition Partition
- 21. Agenda • 介绍 – 基础架构,四要素,内容和相关文档 • Availability(可靠性) • Secure Separation(安全隔离) • Service Assurance(服务保证) • Management (可管理性) • Storage in SMT Architecture(SMT架构中的存储系统) • Summary(总结)
- 22. 服务保证 – 交付服务级别协议(SLA) 计算 Expandable Reservation Dynamic Resource Scheduler UCS QoS System Classes for Resource Reservation and Limit 网络 QoS - Classification QoS - Queuing 2 GE Gold 4 GE Platinum QoS - Bandwidth control CoS CoS QoS - Rate Limiting 存储 FlexShare Storage Reservations Med Priority High Priority Thin Provisioning
- 23. 网络服务保证 Traffic Types Service-Class CoS & UCS Class • QoS – Classification NFS Data Store/N1KV Control & CoS 5 Platinum – Classification Capability Network Management CoS 6 Gold Management vMotion – Identify Traffic Types CoS 4, Silver Transactional CoS 5, Platinum – Classify at Source of Front End Traffic CoS 6, Gold Origin Bulk Data CoS 4, Silver Application CoS 2, Bronze • QoS – Queuing Storage IO Back End Traffic CoS 5, Platinum – Packet Delivery Schedule App to App CoS 6, Gold (multi-tier) CoS 4, Silver • QoS - Bandwidth Control Best Effort Best Effort • QoS – Rate Limiting Scavenger CoS 0 & 1, Best Effort
- 24. 计算资源服务保证 • 内嵌vCenter Resource Pool设置提供: – resource guarantee for infrastructure and tenant services • Resource pool设置基于用户所需的SLA要求: Resource Pool Platinum Tenant Gold Silver Tenant Settings Tenant Reservation Reserved Reserved No reservation Limits Unlimited Limited Limited Shares High Medium Low Expandable Enabled Disabled Disabled Reservation • VMware DRS 提供了一个ESX集群内的完全自动的跨UCS刀片的负载分配 – 在VM/vApp运行状态下 – 在稳定或不稳定的状况下
- 25. 存储系统服务级别协议(SLA)保证 • 可对优先应用或用户提供 Clients 高优先级访问保证 Database Server • Five levels of prioritization Switch available Platinum SLA • 隔离租赁单元性能问题, Gold SLA 保证其它租赁单元的SLA 不受其影响 High Medium Priority Priority 运行Flexshare软件的FAS存储系统
- 26. Agenda • 介绍 – 基础架构,四要素,内容和相关文档 • Availability(可靠性) • Secure Separation(安全隔离) • Service Assurance(服务保证) • Management (可管理性) • Storage in SMT Architecture(SMT架构中的存储系统) • Summary(总结)
- 27. 端到端的管理 Server Layer Unified Computing System vCenter Server (UCS) Manager vShield Manager SANscreen Network Layer Data Center Network Manager Flexible NetFlow SANscreen Fabric Manager Storage Layer Operations Manager Provisioning and Protection Manager SANscreen Service Insight Service Assurance Application Insight Capacity Manager VM Insight
- 28. Cisco UCS Manager • Single point of management for UCS GUI Custom Portal or Tools system of components – Adapters, blades, chassis, fabric extenders, fabric interconnects CLI Systems Management Software • Embedded device manager –Discovery, Inventory, Configuration, Monitoring, Diagnostics, Statistics Collection UCS Manager –Coordinated deployment to managed endpoints • APIs for integration with new and existing data center infrastructure –SMASH-CLP, IPMI, SNMP –XML-based SDK for commercial & custom implementations
- 29. Cisco Data Center Network Manager Centralized management throughout the data center network Ethernet, IP routing and Network Security domain awareness Enables error-free provisioning Configuration validation via syntax and semantics checks Health monitoring Real-time alarms and key traffic performance indicators Facilitates the insertion of innovative network features Network virtualization transparently supported Powerful industry-standard SOAP/XML API Stateful network information enabling network-aware 3rd party applications
- 30. vCenter基础架构管理 Centralized Control and Resource Allocation Overview Visibility Performance Charts Overview Datastore Utilization Overview Proactive Management Default Alarms to monitor infrastructure health, resource and space utilization Extensibility vShield Manager NetApp Virtual Storage Console (VSC)
- 31. vShield Manager 和vCenter server集成 Policy Overview Traffic flow Historical flowchart Real Time flowchart 32
- 32. NetApp存储系统管理 SANscreen allows providers and tenants visibility into full storage path Provisioning Manager eases providers deployment Protection manager makes backups and recovery a snap. Operations Manager offers chargeback reporting and monitoring
- 33. Agenda • 介绍 – 基础架构,四要素,内容和相关文档 • Availability(可靠性) • Secure Separation(安全隔离) • Service Assurance(服务保证) • Management (可管理性) • Storage in SMT Architecture(SMT架构中的存储系统) • Summary(总结)
- 34. SMT架构下的存储系统 • 弹性的存储架构 • 数据可靠性 • 存储有效性 • 虚拟化环境集成 35
- 35. 弹性的存储架构:统一的存储架构设计 SAN NAS Enterprise Departmental Enterprise Departmental iSCSI Fibre Dedicated Corporate Channel Ethernet LAN FCoE SAN NAS (Block) LUN LUN File File (File) FAS Series 36
- 36. 弹性的存储架构:持续在线的数据流动性 解决无法为下列操作安排停机 时间的问题 – 存储扩容 – 计划内维护宕机时间 – 技术更新 – 软件升级 Storage Pool 提高服务级别的灵活度 Storage Pool – 动态的负载均衡 – 可调整的存储设备级别 应用透明化集成 Data Data – 保持优异性能 Data – 保持交易一致性 37 NetApp Confidential - Internal Use Only 37
- 37. 弹性的存储架构:存储资源Qos管理，优化应用响应 High Priority Volumes Medium Priority • 基于卷的优先级定义 • 在线动态调整 Volumes Low Priority Volumes 0 10 20 30 40 50 60 70 80 • 保证关键业务的I/O优先级响 Latency (msec) 应 • 实现存储整合场景下，多应用 下的性能负荷优化 High Priority Volumes Medium Priority Volumes Low Priority Volumes 0 10 20 30 40 50 60 70 80 Latency (msec)
- 38. 数据可靠性:瞬间备份,无服务器性能影响 传统磁带备份 NetApp 快照备份 高服务器利用率 服务器运行应用，没有后台处理 备份窗口小 瞬间数据备份和恢复 磁带备份速度慢/复杂/昂贵/可靠 低存储负荷 性差 应用一致性 容灾困难 VM1 VM2 VM1 VM2 CPU Utilization CPU Utilization snapshot Storage Pool 传统备份方式不再适合云计算 快速, 简单备份/恢复
- 39. 数据可靠性:高效率D2D备份架构 本地或远程站点 优势 Windows/Linux ESX 管理服务器 测试主机 – 备份速度快 OSSV OSSV – 备份窗口小 – 备份/恢复的可靠性高 – 可与远程复制技术联动实现异 地容灾 – 备份或容灾数据可近实时检验 内置磁盘或非NetApp OSSV 存储 FlexClone Copies Snapshot SnapVault SnapMirror NetApp NetApp NetApp 主存储 二级近线存储 异地容灾存储
- 40. 存储有效性:多种空间有效性存储技术 Save Deduplication Save up to over SnapshotTM Copies 95% 重复数据删除 80% 快照备份 Saves up to 95% for full backups; 25% to 55% for most data sets. Up to Save 46% RAID 6 (RAID-DPTM) over Writable Snapshot Saves up to 46% versus mirrored 80% (FlexClone®) Copies data or RAID 10. 虚拟克隆技术 33% Thin Provisioning (FlexVol®) PAM /PAMII 自动精简部署技术 性能加速模块 20% to 33% typical savings. Savings compound when using 41 multiple features!
- 41. 虚拟化环境集成:SMVI虚拟化套件 主站点 容灾站点 APP APP APP Virtual Server OS OS OS Admin API VM VM VM VMDK Storage Pool VMDK VMDK Storage Pool VM1 VMDK 42 42
- 42. Agenda • 介绍 – 基础架构,四要素,内容和相关文档 • Availability(可靠性) • Secure Separation(安全隔离) • Service Assurance(服务保证) • Management (可管理性) • Storage in SMT Architecture(SMT架构中的存储系统) • Summary(总结)
- 43. SMT架构与VCE架构的对比 Secure Multi-Tenancy Virtual Computing Environment 底层存储架构 建立在统一存储架构之上的云计算 主要是建立在SAN存储架构上的云计算 Vblock0/Vblock1/Vblock2三种固定配置（Vblock0暂时 配置组合 无固定配置，可根据实际需要选择不同级别的产品 未发布） Vblock1采用CX4-480（NS-G2为可选）； 适用存储类型 NetApp全系列存储产品 Vblock2采用Symmetrix Vmax(NS-G8网关可选)； Vblock1: Min:CX480/6*SSD/83*FC Disk/21*SATA Max:CX480/6*SSD/151*FC Disk/27*SATA 估计可用容量38-64TB Vblock2: Min:Vmax/9*SSD/124*FC Disk/76*SATA Max:Vmax/9*SSD/240*FC Disk/110*SATA 配置组合 组合灵活 估计可用容量96-146TB FC SAN:ESX操作系统SAN Boot用； FC SAN:ESX操作系统SAN Boot用； NFS NAS:Datastore存储协议 ISCSI:Datastore存储协议 存储协议 iSCSI(Option) NAS(Option):作为VM上应用的补充 数据分级管理 基于SSD的PAM技术实现数据的智能调度 FAST(全自动存储分层)实现LUN级别的存储分层 存储架构扩展 支持纵向、横向线性扩展 存储网络将存在三种不同类别的存储系统 存储网络可能存在三种不同类别的存储系统，管理将 存储系统可管理性 一套存储OS，维护管理简单 复杂 NetApp提供云存储架构下的数据流动性功能- 数据流动性 DataMotion，数据的迁移无需进行服务的中断 由于存在三种存储系统,未提供类似功能 租赁模式支持 是一种充分考虑租赁业务模式的云计算架构 是一种提供主机虚拟化的硬件架构 44
- 44. Q&A