SlideShare a Scribd company logo

PKS: The What and How of Enterprise-Grade Kubernetes

VMware Tanzu, profile picture
VMware Tanzu

The document discusses enterprise-grade Kubernetes and the issues related to operationalizing it, such as high availability and scaling challenges. It outlines the roles of BOSH and Pivotal Container Service (PKS) in simplifying Kubernetes deployment and management, emphasizing self-service clusters, automated operations, and multi-cloud support. Additionally, it highlights the importance of consistent operational experiences across different cloud environments.

Technology
1 of 62
Downloaded 152 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
PKS: The What and How of Enterprise-Grade Kubernetes (or: Because the Cool Kids All Spell Kontainer with a K ;-)) Cornelia Davis, Sr. Director of Technology, Pivotal, @cdavisafc Fred Melo, Director of Technology, Pivotal, @fredmelo_br 1
Disclaimer
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Safe Harbor Statement The following is intended to outline the general direction of Pivotal's offerings. It is intended for information purposes only and may not be incorporated into any contract. Any information regarding pre-release of Pivotal offerings, future updates or other planned modifications is subject to ongoing evaluation by Pivotal and is subject to change. This information is provided without warranty or any kind, express or implied, and is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions regarding Pivotal's offerings. These purchasing decisions should only be based on features currently available. The development, release, and timing of any features or functionality described for Pivotal's offerings in this presentation remain at the sole discretion of Pivotal. Pivotal has no obligation to update forward looking information in this presentation. 3
The Why
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Companies have Many Types of Workloads 5 CONTAINERS EVENT-DRIVEN FUNCTIONS DATA SERVICES MICROSERVICES Batches MONOLITHIC APPLICATIONS
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ …And Different SDLCs 6 “Stable” maintenance Active Development RUN BUILD + RUN
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ CONFIDENTIAL 7 Monolithic PAAS Cloud Native RUN BUILD + RUN
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ CONFIDENTIAL 8 Traditional IaaS Cloud-native App Platform Monolithic PAAS Cloud Native RUN BUILD + RUN
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ CONFIDENTIAL 9 Infrastructure Orchestrated CaaS Cloud-native App Platform Monolithic PAAS Cloud Native RUN BUILD + RUN Traditional IaaS
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ PCF2.0 CONFIDENTIAL 10 Infrastructure Orchestrated CaaS Cloud-native App Platform Monolithic PAAS Cloud Native RUN BUILD + RUN Traditional IaaS
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ CONFIDENTIAL 11 Infrastructure Traditional IaaS Orchestrated CaaS Cloud-native App Platform Monolithic PAAS Cloud Native PCF2.0 RUN BUILD + RUN
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 12 Hardware IaaS Container Orchestrator Application Platform Serverless Functions Strategic goal: Push as many workloads as technically feasible to the top of the platform hierarchy Higher flexibility and less enforcement of standards Lower development complexity and higher operational efficiency
The What
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 14
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Operational Challenges with any platform 15 Patches Patching platform components with thousands of apps running should feel normal. 
 Scaling Seamlessly scale platform components to accommodate changing demand. 
 Upgrades. How do you roll out new versions of the platform with the lights on? Operating Effort Operating a platform should require very few resources and minimum manual intervention. Otherwise, is it really providing operational benefits? Multi-cloud Provide a reliable and smooth experience for any cloud. Open APIs Allow platform operations from different toolsets and the creation of CD pipelines. 
 Consistency Provide a consistent setup experience, across different cloud environment configurations. 
 Setup time How long does it take to setup a real world working environment? Think hours, not weeks. 
 Day 1 - Build Day 2 - Operate
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Kubernetes - especially hard to operationalize 16 High Availability. No out-of-the-box fault- tolerance for the cluster components themselves (masters and etcd nodes). 
 Scaling. Kubernetes clusters handle scaling the pod/service within the Nodes, but doesn’t provide a mechanism to scale Masters & etcd VMs. 
 Health checks and healing. The Kubernetes cluster does routine health checks for the health of Nodes only. 
 Upgrades.  Rolling upgrades on a large fleet of clusters is hard. Who manages the system it runs on?
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ PIVOTAL CLOUD FOUNDRY OPS Powered by BOSH 17 BOSH is an open source tool for release engineering, deployment, lifecycle management, and monitoring of distributed systems. Packaging w/ embedded OS Server provisioning on any IaaS Software deployment across availability zones Health monitoring (server AND processes) Self-healing w/ Resurrector Storage management Rolling upgrades via canaries Easy scaling of clusters
Workeretcd etcd Kubernetes etcd Master Master Worker Worker Packaging w/ embedded OS Server provisioning on any IaaS Software deployment across availability zones Health monitoring (server AND processes) Self-healing w/ Resurrector Storage management Rolling upgrades via canaries Easy scaling of clusters
Project Kubo Uniform way to instantiate, deploy, and manage highly available Kubernetes clusters. On any cloud. Launched by Pivotal & Google Feb 2017, Donated to Cloud Foundry Foundation June 2017 Committers: Pivotal, Google, VMware “Day 1” Build " Deploy Kubernetes cluster via BOSH “Day 2” Operate " Self-healing VMs and monitoring via BOSH " Elastic scaling for clusters " Rolling upgrades to latest Kubernetes release " High-availability and multi-AZ support
Workeretcd etcd Kubernetes etcd Master Master Worker Worker Kubo Provides Specification of K8S Components 20 Platform team is then responsible for assembly into desired clustersRelease templates Manifest Kubo Release bosh deploy
Pivotal Container Service (PKS) Provides the control plane for provisioning and managing Kubo releases Joint development effort between Pivotal and VMWare Kubernetes Dial Tone: • Health management • Framework for Metrics and Logging • Autoscaling • Persistence interface • Networking interface Control Plane: • Provisioning Engine • Self-service Clusters • Software Update Automation • Load balancing • Networking • Persistence • Multi-tenancy
PKS: Provisioning Engine 22 PKS Service Broker Release templates Manifest Kubo Release
23 PKS Service Broker Release templates Manifest Kubo Release create cluster (with upgrade policy) PKS: Self-service Clusters
24 PKS: Networking (different options available) BOSH network Worker Worker 10.0.30.12 ContainerContainerContainer 10.200.2.6C2C overlay 10.200.1.510.200.1.4 10.0.30.11 kube-proxy iptables kube-proxy iptables Service network Service
BOSH GCP
 Service
 Broker Harbor NSX-T Kubernetes K8s Cluster K8s Cluster K8s Cluster Built with open-source Kubernetes — Constant compatibility with the current stable release of Kubernetes, operated by BOSH. No proprietary extensions. 
 Production-ready — Highly available from apps to infrastructure, no single points of failure. Built-in health checks, scaling, auto-healing and rolling upgrades. 
 Multicloud — BOSH provides a reliable and consistent operational experience. For any cloud. Network management and security out-of-the-box with VMware NSX-T. Multi-cloud, multi-hypervisor. GCP APIs access — The GCP Service Broker allows apps to transparently access Google Cloud APIs, from anywhere. Easily move workloads to/from Google Container Engine (GKE). 
 Fully automated Ops — Fully automated deploy, scale, patch, upgrade. No downtime. Use CD pipelines to deploy your platform, too. VMware GCP Azure Openstack AWS PKSController
Apps are constantly compatible between PKS and Google Container Engine (GKE) Google Container Engine (GKE) Pivotal Container Service (PKS) Private and Public Cloud Public Cloud (on GCP)
VMware PKS Analytics Automation SecurityOperations MonitoringLogging Physical Infrastructure Container Registry vSphere vSAN Kubernetes on BOSH (Kubo) NSX GCP Service 
 Broker masteretcd workermasteretcd worker
Leveraging more than one abstraction Other Broker Services Platform Services Logging Metrics Monitoring Pivotal Application Service Application Application VMware GCP Azure Openstack AWS PKSController GCP
 Service
 Broker Harbor NSX-T Kubernetes K8s Cluster K8s Cluster K8s Cluster
Unified Platform Experience Shared Logging and Metrics Shared Networking Shared Security
The How
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 31 BOSH GCP
 Service
 Broker Harbor NSX-T Kubernetes K8s Cluster K8s Cluster K8s Cluster Built with open-source Kubernetes — Constant compatibility with the current stable release of Kubernetes, operated by BOSH. No proprietary extensions. 
 Production-ready — Highly available from apps to infrastructure, no single points of failure. Built-in health checks, scaling, auto-healing and rolling upgrades. 
 Multicloud — BOSH provides a reliable and consistent operational experience. For any cloud. Network management and security out-of-the-box with VMware NSX-T. Multi-cloud, multi-hypervisor. GCP APIs access — The GCP Service Broker allows apps to transparently access Google Cloud APIs, from anywhere. Easily move workloads to/from Google Container Engine (GKE). 
 Fully automated Ops — Fully automated deploy, scale, patch, upgrade. No downtime. Use CD pipelines to deploy your platform, too. VMware GCP Azure Openstack AWS PKSController How does this all work? • Availability and Resiliency • Operational Efficiency • Multi-tenancy isolation • Security • Extensibility Guiding Principles
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 32 BOSH PKSControlPlane Pivotal Container Service
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ BOSH Pivotal Container Service PKSControlPlane Create cluster
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 34 Availability Zone B Availability Zone A HA and Health Management - Kubelet watches and restart containers - Bosh director watches and restarts nodes - Bosh agent watches and restarts processes - Bosh distributes deployments across AZ’s Kubelet Kube-proxy Pod Pod K8s Node Pod API Server Kube Scheduler K8s Master Controller Manager Bosh agent Bosh agentBosh director Watches and restarts VMs Availability Zone A Availability Zone B Create cluster
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 35 BOSH Pivotal Container ServicePKSControlPlane Kubelet Kube-proxy Fluentd Pod Pod Pod K8s Node Kubelet Kube-proxy Fluentd Pod Pod Pod K8s Node Kubelet Kube-proxy Pod Pod Pod K8s Worker API Server Kube Scheduler K8s Master Controller Manager Node Controller Replication Controller Endpoints Controller Service Accounts & Token Etcd GCP
 Service
 BrokerHarbor NSX-T Proxy NCP T1 Routers NSX-T T1 Routers Other components Included component, but optional usage CFCR / Kubo
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 36 BOSH Pivotal Container ServicePKSControlPlane Kubelet Kube-proxy Fluentd Pod Pod Pod K8s Node Kubelet Kube-proxy Fluentd Pod Pod Pod K8s Node Kubelet Kube-proxy Pod Pod Pod K8s Worker API Server Kube Scheduler K8s Master Controller Manager Node Controller Replication Controller Endpoints Controller Service Accounts & Token Etcd GCP
 Service
 BrokerHarbor NSX-T Proxy NCP T1 Routers NSX-T T1 Routers Other components Included component, but optional usage CFCR / Kubo Kubernetes
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Kubernetes components API Server Kube Scheduler K8s Master Controller Manager Etcd K-V Store Kubelet Kube-proxy K8s Worker Pod Pod Pod Kubelet Kube-proxy K8s Worker Pod Pod Pod Kubelet Kube-proxy K8s Worker Pod Pod Pod CNI Persistent Volume Virtual switch Network edge router Volume mount Aws, Azure, GCP, vSphere SCSI, FC, NFS, Photon, Scale IO, … Provisioner NSX-T, Flannel, Calico, GCE, Nuange, OVN, Kube-router… CNI Plugin Protobuf Network edge routerNetwork edge router
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 38 API Server Kube Scheduler K8s Master Controller Manager Etcd K-V Store Kubelet Kube-proxy K8s Worker Pod Pod Pod Kubelet Kube-proxy K8s Worker Pod Pod Pod Kubelet Kube-proxy K8s Worker Pod Pod Pod CNI Persistent Volume Virtual switch Network edge router Volume mount Aws, Azure, GCP, vSphere SCSI, FC, NFS, Photon, Scale IO, … Provisioner NSX-T, Flannel, Calico, GCE, Nuage, OVN, Kube-router… CNI Plugin Protobuf Network edge routerNetwork edge router Pluggable Networking (CNI)
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 39 BOSH Pivotal Container Service PKSController GCP
 Service
 Broker BOSH network 10.0.1.5 Harbor 10.0.1.2 Etcd 10.0.1.6 Etcd 10.0.1.6 API Server Kube Scheduler K8s Master Controller Manager Node Controller Replication Controller Endpoints Controller Service Accounts & Token 10.0.1.4 Kubelet Kube-proxy Fluentd Pod Pod Pod K8s Node 10.0.1.3 Kubelet Kube-proxy Fluentd Pod Pod Pod K8s Node 10.0.1.3 Pod Pod Pod K8s Worker Kubelet Kube-proxy 10.0.1.3 Networking
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 40 BOSH PKSController GCP
 Service
 Broker Harbor Etcd API Server Kube Scheduler K8s Master Controller Manager Node Controller Replication Controller Endpoints Controller Service Accounts & Token Base Network 10.0.0.11 10.0.0.16 10.0.0.17 10.0.0.13 BOSH Network NSX-T Mgr NSX-T NCP .18 .19 10.0.0.21 10.0.0.22 Kubelet NSX Kube-Proxy OVS K8s Worker K8s Worker Kubelet
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 41 BOSH PKSController GCP
 Service
 Broker Harbor Etcd API Server Kube Scheduler K8s Master Controller Manager Node Controller Replication Controller Endpoints Controller Service Accounts & Token Base Network 10.0.0.11 10.0.0.16 10.0.0.17 10.0.0.13 BOSH Network NSX-T Mgr NSX-T NCP .18 .19 10.0.0.21 10.0.0.22 Kubelet Pod Pod NSX Kube-Proxy OVS K8s Worker Pod Pod NSX Kube-Proxy K8s Worker K8K8s NameSpace A = NSX Logical Switch K8K8s NameSpace B = NSX Logical Switch T1 10.172.10.2 10.172.100.0/24 192.168.100.0/24 NO_NAT NAT T1 10.172.10.3 T0 10.172.10.1 Kubelet OVS NSX CNI PLUGIN Overlay Networks
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 42 -T Pivotal Container Service Network Policy Management Multi-Tenancy Isolation Network Security Groups Dynamic Load Balancing & Ingress Comprehensive VM + Container security model
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 43 kubectl apply -f <file.yml> Network Policies Management BOSH Pivotal Container Service PKSController NSX-T Proxy NCP T1 RoutersT1 Routers Other components NSX-T Kubelet Kube- proxy Fluentd Pod Pod Pod K8s Node Kubelet Kube- proxy Fluentd Pod Pod Pod K8s NodeNode NSX Router Network policy management API Server Kube Scheduler K8s Master Controller Manager Etcd new policy Apply Kubelet K. Proxy Pod Pod Pod
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 44 Deployment Topologies & Multi-Tenancy Multi-cluster Single cluster K8s Cluster A K8s Cluster BOSH PKSController Namespace A Namespace B Namespace C BOSH PKSController K8s Cluster B K8s Cluster C
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 45 admin@k8s-master:~$ kubectl create namespace foo namespace ”foo" created admin@k8s-master:~$ kubectl create namespace bar namespace ”bar" created admin@k8s-master:~$ kubectl run nginx-foo --image=nginx -n foo deployment "nginx-foo" created admin@k8s-master:~$ kubectl run nginx-bar --image=nginx -n bar deployment "nginx-bar" created Namespace: foo Namespace: bar NSX / K8s topology 10.24.0.0/24 10.24.1.0/24 10.24.2.0/24 NAT boundary NAT boundary K8s nodesK8s Masters NameSpace as a Tenancy Construct NSX-T Automatically Provides Each Namespace a secure network & routing
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 46 admin@k8s-master:~$ kubectl label pods nginx-foo-3492604561-nltrf secgroup=web -n foo Pod "nginx-nsx-3492604561-nltrf" labeled admin@k8s-master:~$ kubectl label pods nginx-bar-2789337611-z09x2 secgroup=db -n bar pod "nginx-k8s-2789337611-z09x2" labeled admin@k8s-master:~$ kubectl get pods --all-namespaces -Lsecgroup NAMESPACE NAME READY STATUS RESTARTS AGE SECGROUP k8s nginx-foo-2789337611-z09x2 1/1 Running 0 58m web nsx nginx-bar-3492604561-nltrf 1/1 Running 0 1h db Namespace: foo Namespace: bar NSX / K8s topology 10.24.0.0/24 10.24.1.0/24 114.4.10.0/26 NAT boundary NAT boundary Web • Security Groups are defined in NSX with ingress and egress policy • Each Security Group could be micro-segmented to protect Pods from each other DB
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 47 App / Container Deployment, Services & Routing cf push App myapp.mydomain.net Pivotal Cloud Foundry Application Runtime Pivotal Container Service kubectl run ???? ????
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 48 App / Container Deployment, Services & Routing cf push App myapp.mydomain.net Pivotal Cloud Foundry Application Runtime Pivotal Container Service kubectl run ???? ????
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 49 BOSH Pivotal Container Service PKSController Kubelet Kube-proxy Pod K8s Worker K8s Master Controller Manager Node Controller Replication Controller Endpoints Controller Service Accounts & Token kubectl run <image> Harbor Etcd Pod API Server Kube Scheduler Pod Pod
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 50 ??? kubectl run <docker img> K8s Worker Kubelet Kube-proxy Pod Pod Pod 192.168.0.4 192.168.0.3 192.168.0.2 API Server Kube Scheduler K8s Master Controller Manager API Server Kube Scheduler K8s Master Controller Manager K8s Worker Kubelet Kube-proxy Pod Pod Pod 192.168.0.4 192.168.0.3 192.168.0.2 K8s Worker Kubelet Kube-proxy Pod Pod 192.168.0.4 192.168.0.3 Pod 192.168.0.2 K8s Cluster Base Network Overlay Network Cluster network (conceptual)
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 51 kubectl run <docker img> K8s Worker Kubelet Kube-proxy Pod Pod Pod 192.168.0.4 192.168.0.3 192.168.0.2 Cluster network (conceptual) API Server Kube Scheduler K8s Master Controller Manager API Server Kube Scheduler K8s Master Controller Manager K8s Cluster K8s Worker Kubelet Kube-proxy Pod Pod Pod 192.168.0.4 192.168.0.3 192.168.0.2 Base Network Overlay Network Service K8s Worker Kubelet Kube-proxy Pod Pod 192.168.0.4 192.168.0.3 Pod 192.168.0.2
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 52 NodePort K8s Worker Kubelet Kube-proxy Pod Pod Pod 192.168.0.4 192.168.0.3 192.168.0.2 Cluster network (conceptual) API Server Kube Scheduler K8s Master Controller Manager API Server Kube Scheduler K8s Master Controller Manager K8s Cluster K8s Worker Kubelet Kube-proxy Pod Pod Pod 192.168.0.4 192.168.0.3 192.168.0.2 Base Network Overlay Network kubectl expose deployment NodePort NodePort K8s Worker Kubelet Kube-proxy Pod Pod 192.168.0.4 192.168.0.3 Pod 192.168.0.2 host:port NodePort Load Balancer host
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 53 NodePort K8s Worker Kubelet Kube-proxy Pod Pod Pod 192.168.0.4 192.168.0.3 192.168.0.2 API Server Kube Scheduler K8s Master Controller Manager API Server Kube Scheduler K8s Master Controller Manager K8s Worker Kubelet Kube-proxy Pod Pod Pod 192.168.0.4 192.168.0.3 192.168.0.2 kubectl expose deployment Load Balancer NodePort K8s Worker Kubelet Kube-proxy Pod Pod 192.168.0.4 192.168.0.3 Pod 192.168.0.2 NodePort Cloud Load Balancer host Dynamically provisioned Kubernetes Cluster Cloud 35.190.151.218 Public IP -T
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 54 Services: Ingress An API object that manages external access to the services in a cluster, typically HTTP. Ingress can provide load balancing, SSL termination and name-based virtual hosting. K8s Worker Kubelet Kube-proxy PodPodPod K8s Worker Kubelet Kube-proxy PodPodPodS2 S1 K8s Worker Kubelet Kube-proxy PodPod Pod Ingress /foo /bar foo.bar.com Deploy
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 5 BOSH Pivotal Container Service PKSController Kubelet Kube-proxy Fluentd Pod Pod K8s Node API Server Kube Scheduler K8s Master Controller Manager Node Controller Replication Controller Endpoints Controller Service Accounts & Token Pod 192.168.0.1 192.168.0.4 192.168.0.3 192.168.0.2 Pivotal Application Service Route-sync GO Router myapp.apps.myorg.net kubectl expose deployment (NodePort w/ route sync) NodePort Integration with CFAR Routing
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 56 API Server Kube Scheduler K8s Master Controller Manager Etcd K-V Store Kubelet Kube-proxy K8s Worker Pod Pod Pod Kubelet Kube-proxy K8s Worker Pod Pod Pod Kubelet Kube-proxy K8s Worker Pod Pod Pod CNI Persistent Volume Virtual switch Network edge router Volume mount Aws, Azure, GCP, vSphere SCSI, FC, NFS, Photon, Scale IO, … Provisioner Protobuf Network edge routerNetwork edge router Pluggable Persistent Volumes
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 57 Pod Container Container Host mount Fiber Channel, iSCSI, NFS GCP, Azure, AWS, vSphere Volumes and Persistent Volumes Legacy Systems Stateful Systems Database
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 58 kubectl apply -f <file.yml> Pivotal Container Service PKSController K8s Master Etcd Worker Kubelet K. Proxy Pod Pod Worker Kubelet K. Proxy Pod Pod Worker Kubelet K. Proxy Pod Pod BOSH
Persistent Volumes
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 60 Stateful Sets Stable, unique network identifiers Stable, persistent storage 
 Ordered, graceful deployment and scaling 
 Ordered, automated rolling updates * stable = survives Pod rescheduling 
 Pod Stateful set Name = web Replicas = 3 Pod web-0 web-1 web-2 Pod PV-claim-web-0 Each Pod gets an unique ID and Persistent Volumes PV-claim-web-1 PV-claim-web-2
Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 61 Stateful Sets Apps / clusters using specific IPs / names to communicate Apps / clusters relying on persistent and durable storage on specific mount points Apps / clusters with specific starting order for components Stable, unique network identifiers Stable, persistent storage 
 Ordered, graceful deployment and scaling 
 Ordered, automated rolling updates * stable = survives Pod rescheduling 
 Clustered, stateful workloads or legacy apps
Learn More. Stay Connected. 62 #springone@s1p

More Related Content

PDF
Pivotal Developer-Ready Infrastructure Slides
VMware Tanzu
 
PDF
Cloud-Native Operations with Kubernetes and CI/CD
VMware Tanzu
 
PDF
Pivotal Cloud Foundry 2.5: A First Look
VMware Tanzu
 
PDF
Pivotal Cloud Foundry 2.0: First Look
VMware Tanzu
 
PPTX
Pivotal Container Service Overview
VMware Tanzu
 
PDF
Pivotal Cloud Foundry 2.6: A First Look
VMware Tanzu
 
PDF
Pivotal Cloud Foundry 2.3: A First Look
VMware Tanzu
 
PPTX
Hitting the Enterprise Sweet Spot—A Real-World View of PKS Deployment and Suc...
VMware Tanzu
 
Pivotal Developer-Ready Infrastructure Slides
VMware Tanzu
 
Cloud-Native Operations with Kubernetes and CI/CD
VMware Tanzu
 
Pivotal Cloud Foundry 2.5: A First Look
VMware Tanzu
 
Pivotal Cloud Foundry 2.0: First Look
VMware Tanzu
 
Pivotal Container Service Overview
VMware Tanzu
 
Pivotal Cloud Foundry 2.6: A First Look
VMware Tanzu
 
Pivotal Cloud Foundry 2.3: A First Look
VMware Tanzu
 
Hitting the Enterprise Sweet Spot—A Real-World View of PKS Deployment and Suc...
VMware Tanzu
 

What's hot (20)

PDF
Accelerate Digital Transformation with Pivotal Cloud Foundry on Azure
VMware Tanzu
 
PPTX
PKS Networking with NSX-T: You Focus on your App, We'll Take Care of the Rest!
VMware Tanzu
 
PDF
PCF: Platform for a New Era - Kubernetes for the Enterprise - London
VMware Tanzu
 
PDF
VMware Developer-Ready Transformation
VMware Tanzu
 
PDF
Pivotal Cloud Foundry 2.1: Making Transformation Real Webinar
VMware Tanzu
 
PDF
Pivotal Container Service (PKS) at SF Cloud Foundry Meetup
cornelia davis
 
PPTX
Enterprise pks overview
Boskey Savla
 
PDF
How to Overcome Data Challenges When Refactoring Monoliths to Microservices
VMware Tanzu
 
PDF
How to Scale Operations for a Multi-Cloud Platform using PCF
VMware Tanzu
 
PPTX
Basics of Kubernetes on BOSH: Run Production-grade Kubernetes on the SDDC
Matt McNeeney
 
PPTX
Fabio rapposelli pks-vmug
VMUG IT
 
PDF
Concourse, Spinnaker, Cloud Foundry, Oh My! Creating Sophisticated Deployment...
VMware Tanzu
 
PDF
Migrating from Self-Managed Kubernetes on EC2 to a GitOps Enabled EKS
Weaveworks
 
PDF
PCF Cloud-Native Workshop Slides
VMware Tanzu
 
PDF
Kube Your Enthusiasm - Paul Czarkowski
VMware Tanzu
 
PDF
Pivotal Cloud Foundry 1.10: First Look - Windows at Scale, Network Isolation
VMware Tanzu
 
PDF
클라우드 네이티브 플랫폼의 미래 - Kubernetes 기반의 PCF 로드맵
VMware Tanzu Korea
 
PPTX
Declarative Infrastructure with Cloud Foundry BOSH
cornelia davis
 
PPTX
Netflix MSA and Pivotal
VMware Tanzu Korea
 
PPTX
Deploying Spring Boot apps on Kubernetes
VMware Tanzu
 
Accelerate Digital Transformation with Pivotal Cloud Foundry on Azure
VMware Tanzu
 
PKS Networking with NSX-T: You Focus on your App, We'll Take Care of the Rest!
VMware Tanzu
 
PCF: Platform for a New Era - Kubernetes for the Enterprise - London
VMware Tanzu
 
VMware Developer-Ready Transformation
VMware Tanzu
 
Pivotal Cloud Foundry 2.1: Making Transformation Real Webinar
VMware Tanzu
 
Pivotal Container Service (PKS) at SF Cloud Foundry Meetup
cornelia davis
 
Enterprise pks overview
Boskey Savla
 
How to Overcome Data Challenges When Refactoring Monoliths to Microservices
VMware Tanzu
 
How to Scale Operations for a Multi-Cloud Platform using PCF
VMware Tanzu
 
Basics of Kubernetes on BOSH: Run Production-grade Kubernetes on the SDDC
Matt McNeeney
 
Fabio rapposelli pks-vmug
VMUG IT
 
Concourse, Spinnaker, Cloud Foundry, Oh My! Creating Sophisticated Deployment...
VMware Tanzu
 
Migrating from Self-Managed Kubernetes on EC2 to a GitOps Enabled EKS
Weaveworks
 
PCF Cloud-Native Workshop Slides
VMware Tanzu
 
Kube Your Enthusiasm - Paul Czarkowski
VMware Tanzu
 
Pivotal Cloud Foundry 1.10: First Look - Windows at Scale, Network Isolation
VMware Tanzu
 
클라우드 네이티브 플랫폼의 미래 - Kubernetes 기반의 PCF 로드맵
VMware Tanzu Korea
 
Declarative Infrastructure with Cloud Foundry BOSH
cornelia davis
 
Netflix MSA and Pivotal
VMware Tanzu Korea
 
Deploying Spring Boot apps on Kubernetes
VMware Tanzu
 
Ad

Similar to PKS: The What and How of Enterprise-Grade Kubernetes (20)

PDF
Pivotal Container Service il modo più semplice per gestire Kubernetes in azie...
VMware Tanzu
 
PDF
GPCloud ( GP on PKS)
VMware Tanzu Korea
 
PDF
Pivotal Container Service : la nuova soluzione per gestire Kubernetes in azienda
VMware Tanzu
 
PDF
You've Made Kubernetes Available to Your Developers, Now What?
cornelia davis
 
PDF
Kubernetes for the Spring Developer
VMware Tanzu
 
PDF
Driving Digital Transformation With Containers And Kubernetes Complete Deck
SlideTeam
 
PDF
Running and Managing Kubernetes on OpenStack
Victor Palma
 
PPTX
Infrastructure design for Kubernetes
Guillaume Morini
 
PDF
Developer Secure Containers for the Cyberspace Battlefield
VMware Tanzu
 
PPTX
The Reality of DIY Kubernetes vs. PKS
VMware Tanzu
 
PDF
Kubernetes Concepts And Architecture Powerpoint Presentation Slides
SlideTeam
 
PPTX
TRAINING_ABOUT_KUBERNETES_Nguyen_Si_Nhan.pptx
nhannguyensi
 
PPTX
Successful Patterns for running platforms
Paul Czarkowski
 
PPTX
Containers Orchestration using kubernates.pptx
rani marri
 
PDF
Deploying your first application with Kubernetes
OVHcloud
 
PPTX
Continuous Everything in a Multi-cloud and Multi-platform Environment
VMware Tanzu
 
PDF
Kubernetes and Windows: At Scale with Enterprise PKS
VMware Tanzu
 
DOCX
Kubernetes - An introduction
Loves Cloud
 
PDF
Introduction of Kubernetes - Trang Nguyen
Trang Nguyen
 
PPTX
Introduction to Kubernetes
Paris Apostolopoulos
 
Pivotal Container Service il modo più semplice per gestire Kubernetes in azie...
VMware Tanzu
 
GPCloud ( GP on PKS)
VMware Tanzu Korea
 
Pivotal Container Service : la nuova soluzione per gestire Kubernetes in azienda
VMware Tanzu
 
You've Made Kubernetes Available to Your Developers, Now What?
cornelia davis
 
Kubernetes for the Spring Developer
VMware Tanzu
 
Driving Digital Transformation With Containers And Kubernetes Complete Deck
SlideTeam
 
Running and Managing Kubernetes on OpenStack
Victor Palma
 
Infrastructure design for Kubernetes
Guillaume Morini
 
Developer Secure Containers for the Cyberspace Battlefield
VMware Tanzu
 
The Reality of DIY Kubernetes vs. PKS
VMware Tanzu
 
Kubernetes Concepts And Architecture Powerpoint Presentation Slides
SlideTeam
 
TRAINING_ABOUT_KUBERNETES_Nguyen_Si_Nhan.pptx
nhannguyensi
 
Successful Patterns for running platforms
Paul Czarkowski
 
Containers Orchestration using kubernates.pptx
rani marri
 
Deploying your first application with Kubernetes
OVHcloud
 
Continuous Everything in a Multi-cloud and Multi-platform Environment
VMware Tanzu
 
Kubernetes and Windows: At Scale with Enterprise PKS
VMware Tanzu
 
Kubernetes - An introduction
Loves Cloud
 
Introduction of Kubernetes - Trang Nguyen
Trang Nguyen
 
Introduction to Kubernetes
Paris Apostolopoulos
 
Ad

More from VMware Tanzu (20)

PDF
Spring into AI presented by Dan Vega 5/14
VMware Tanzu
 
PDF
What AI Means For Your Product Strategy And What To Do About It
VMware Tanzu
 
PDF
Make the Right Thing the Obvious Thing at Cardinal Health 2023
VMware Tanzu
 
PPTX
Enhancing DevEx and Simplifying Operations at Scale
VMware Tanzu
 
PDF
Spring Update | July 2023
VMware Tanzu
 
PPTX
Platforms, Platform Engineering, & Platform as a Product
VMware Tanzu
 
PPTX
Building Cloud Ready Apps
VMware Tanzu
 
PDF
Spring Boot 3 And Beyond
VMware Tanzu
 
PDF
Spring Cloud Gateway - SpringOne Tour 2023 Charles Schwab.pdf
VMware Tanzu
 
PDF
Simplify and Scale Enterprise Apps in the Cloud | Boston 2023
VMware Tanzu
 
PDF
Simplify and Scale Enterprise Apps in the Cloud | Seattle 2023
VMware Tanzu
 
PPTX
tanzu_developer_connect.pptx
VMware Tanzu
 
PDF
Tanzu Virtual Developer Connect Workshop - French
VMware Tanzu
 
PDF
Tanzu Developer Connect Workshop - English
VMware Tanzu
 
PDF
Virtual Developer Connect Workshop - English
VMware Tanzu
 
PDF
Tanzu Developer Connect - French
VMware Tanzu
 
PDF
Simplify and Scale Enterprise Apps in the Cloud | Dallas 2023
VMware Tanzu
 
PDF
SpringOne Tour: Deliver 15-Factor Applications on Kubernetes with Spring Boot
VMware Tanzu
 
PDF
SpringOne Tour: The Influential Software Engineer
VMware Tanzu
 
PDF
SpringOne Tour: Domain-Driven Design: Theory vs Practice
VMware Tanzu
 
Spring into AI presented by Dan Vega 5/14
VMware Tanzu
 
What AI Means For Your Product Strategy And What To Do About It
VMware Tanzu
 
Make the Right Thing the Obvious Thing at Cardinal Health 2023
VMware Tanzu
 
Enhancing DevEx and Simplifying Operations at Scale
VMware Tanzu
 
Spring Update | July 2023
VMware Tanzu
 
Platforms, Platform Engineering, & Platform as a Product
VMware Tanzu
 
Building Cloud Ready Apps
VMware Tanzu
 
Spring Boot 3 And Beyond
VMware Tanzu
 
Spring Cloud Gateway - SpringOne Tour 2023 Charles Schwab.pdf
VMware Tanzu
 
Simplify and Scale Enterprise Apps in the Cloud | Boston 2023
VMware Tanzu
 
Simplify and Scale Enterprise Apps in the Cloud | Seattle 2023
VMware Tanzu
 
tanzu_developer_connect.pptx
VMware Tanzu
 
Tanzu Virtual Developer Connect Workshop - French
VMware Tanzu
 
Tanzu Developer Connect Workshop - English
VMware Tanzu
 
Virtual Developer Connect Workshop - English
VMware Tanzu
 
Tanzu Developer Connect - French
VMware Tanzu
 
Simplify and Scale Enterprise Apps in the Cloud | Dallas 2023
VMware Tanzu
 
SpringOne Tour: Deliver 15-Factor Applications on Kubernetes with Spring Boot
VMware Tanzu
 
SpringOne Tour: The Influential Software Engineer
VMware Tanzu
 
SpringOne Tour: Domain-Driven Design: Theory vs Practice
VMware Tanzu
 

Recently uploaded (20)

PDF
Advanced Soft Computing BINUS July 2025.pdf
University of Hertfordshire
 
PDF
Machine learning based COVID-19 study performance prediction
IAESIJAI
 
PDF
Reach Out and Touch Someone: Haptics and Empathic Computing
Mark Billinghurst
 
PDF
GamePlan Trading System Review: Professional Trader's Honest Take
SOFTTECHHUB
 
PDF
Electronic commerce courselecture one. Pdf
OmerMohamed64
 
PDF
cuic standard and advanced reporting.pdf
SimoneTitaniumTomase
 
PPT
Teaching material agriculture food technology
LiaRayya
 
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
Dropbox
 
PDF
Approach and Philosophy of On baking technology
30anthuong17
 
PDF
KodekX | Application Modernization Development
KodekX
 
PDF
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Akar16
 
PDF
Modernizing your data center with Dell and AMD
Principled Technologies
 
PDF
NewMind AI Weekly Chronicles - August'25 Week I
NewMind AI
 
PPT
“AI and Expert System Decision Support & Business Intelligence Systems”
ZubinRadhakrishnan
 
PDF
Advanced methodologies resolving dimensionality complications for autism neur...
IAESIJAI
 
PDF
Bridging biosciences and deep learning for revolutionary discoveries: a compr...
IAESIJAI
 
PDF
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
SOFTTECHHUB
 
PDF
Empathic Computing: Creating Shared Understanding
Mark Billinghurst
 
PDF
solutions_manual_-_materials___processing_in_manufacturing__demargo_.pdf
AbdullahSani29
 
PDF
Network Security Unit 5.pdf for BCA BBA.
Serpent6
 
Advanced Soft Computing BINUS July 2025.pdf
University of Hertfordshire
 
Machine learning based COVID-19 study performance prediction
IAESIJAI
 
Reach Out and Touch Someone: Haptics and Empathic Computing
Mark Billinghurst
 
GamePlan Trading System Review: Professional Trader's Honest Take
SOFTTECHHUB
 
Electronic commerce courselecture one. Pdf
OmerMohamed64
 
cuic standard and advanced reporting.pdf
SimoneTitaniumTomase
 
Teaching material agriculture food technology
LiaRayya
 
Dropbox Q2 2025 Financial Results & Investor Presentation
Dropbox
 
Approach and Philosophy of On baking technology
30anthuong17
 
KodekX | Application Modernization Development
KodekX
 
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Akar16
 
Modernizing your data center with Dell and AMD
Principled Technologies
 
NewMind AI Weekly Chronicles - August'25 Week I
NewMind AI
 
“AI and Expert System Decision Support & Business Intelligence Systems”
ZubinRadhakrishnan
 
Advanced methodologies resolving dimensionality complications for autism neur...
IAESIJAI
 
Bridging biosciences and deep learning for revolutionary discoveries: a compr...
IAESIJAI
 
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
SOFTTECHHUB
 
Empathic Computing: Creating Shared Understanding
Mark Billinghurst
 
solutions_manual_-_materials___processing_in_manufacturing__demargo_.pdf
AbdullahSani29
 
Network Security Unit 5.pdf for BCA BBA.
Serpent6
 

PKS: The What and How of Enterprise-Grade Kubernetes

  • 1. PKS: The What and How of Enterprise-Grade Kubernetes (or: Because the Cool Kids All Spell Kontainer with a K ;-)) Cornelia Davis, Sr. Director of Technology, Pivotal, @cdavisafc Fred Melo, Director of Technology, Pivotal, @fredmelo_br 1
  • 3. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Safe Harbor Statement The following is intended to outline the general direction of Pivotal's offerings. It is intended for information purposes only and may not be incorporated into any contract. Any information regarding pre-release of Pivotal offerings, future updates or other planned modifications is subject to ongoing evaluation by Pivotal and is subject to change. This information is provided without warranty or any kind, express or implied, and is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions regarding Pivotal's offerings. These purchasing decisions should only be based on features currently available. The development, release, and timing of any features or functionality described for Pivotal's offerings in this presentation remain at the sole discretion of Pivotal. Pivotal has no obligation to update forward looking information in this presentation. 3
  • 5. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Companies have Many Types of Workloads 5 CONTAINERS EVENT-DRIVEN FUNCTIONS DATA SERVICES MICROSERVICES Batches MONOLITHIC APPLICATIONS
  • 6. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ …And Different SDLCs 6 “Stable” maintenance Active Development RUN BUILD + RUN
  • 7. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ CONFIDENTIAL 7 Monolithic PAAS Cloud Native RUN BUILD + RUN
  • 8. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ CONFIDENTIAL 8 Traditional IaaS Cloud-native App Platform Monolithic PAAS Cloud Native RUN BUILD + RUN
  • 9. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ CONFIDENTIAL 9 Infrastructure Orchestrated CaaS Cloud-native App Platform Monolithic PAAS Cloud Native RUN BUILD + RUN Traditional IaaS
  • 10. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ PCF2.0 CONFIDENTIAL 10 Infrastructure Orchestrated CaaS Cloud-native App Platform Monolithic PAAS Cloud Native RUN BUILD + RUN Traditional IaaS
  • 11. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ CONFIDENTIAL 11 Infrastructure Traditional IaaS Orchestrated CaaS Cloud-native App Platform Monolithic PAAS Cloud Native PCF2.0 RUN BUILD + RUN
  • 12. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 12 Hardware IaaS Container Orchestrator Application Platform Serverless Functions Strategic goal: Push as many workloads as technically feasible to the top of the platform hierarchy Higher flexibility and less enforcement of standards Lower development complexity and higher operational efficiency
  • 14. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 14
  • 15. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Operational Challenges with any platform 15 Patches Patching platform components with thousands of apps running should feel normal. 
 Scaling Seamlessly scale platform components to accommodate changing demand. 
 Upgrades. How do you roll out new versions of the platform with the lights on? Operating Effort Operating a platform should require very few resources and minimum manual intervention. Otherwise, is it really providing operational benefits? Multi-cloud Provide a reliable and smooth experience for any cloud. Open APIs Allow platform operations from different toolsets and the creation of CD pipelines. 
 Consistency Provide a consistent setup experience, across different cloud environment configurations. 
 Setup time How long does it take to setup a real world working environment? Think hours, not weeks. 
 Day 1 - Build Day 2 - Operate
  • 16. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Kubernetes - especially hard to operationalize 16 High Availability. No out-of-the-box fault- tolerance for the cluster components themselves (masters and etcd nodes). 
 Scaling. Kubernetes clusters handle scaling the pod/service within the Nodes, but doesn’t provide a mechanism to scale Masters & etcd VMs. 
 Health checks and healing. The Kubernetes cluster does routine health checks for the health of Nodes only. 
 Upgrades.  Rolling upgrades on a large fleet of clusters is hard. Who manages the system it runs on?
  • 17. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ PIVOTAL CLOUD FOUNDRY OPS Powered by BOSH 17 BOSH is an open source tool for release engineering, deployment, lifecycle management, and monitoring of distributed systems. Packaging w/ embedded OS Server provisioning on any IaaS Software deployment across availability zones Health monitoring (server AND processes) Self-healing w/ Resurrector Storage management Rolling upgrades via canaries Easy scaling of clusters
  • 18. Workeretcd etcd Kubernetes etcd Master Master Worker Worker Packaging w/ embedded OS Server provisioning on any IaaS Software deployment across availability zones Health monitoring (server AND processes) Self-healing w/ Resurrector Storage management Rolling upgrades via canaries Easy scaling of clusters
  • 19. Project Kubo Uniform way to instantiate, deploy, and manage highly available Kubernetes clusters. On any cloud. Launched by Pivotal & Google Feb 2017, Donated to Cloud Foundry Foundation June 2017 Committers: Pivotal, Google, VMware “Day 1” Build " Deploy Kubernetes cluster via BOSH “Day 2” Operate " Self-healing VMs and monitoring via BOSH " Elastic scaling for clusters " Rolling upgrades to latest Kubernetes release " High-availability and multi-AZ support
  • 20. Workeretcd etcd Kubernetes etcd Master Master Worker Worker Kubo Provides Specification of K8S Components 20 Platform team is then responsible for assembly into desired clustersRelease templates Manifest Kubo Release bosh deploy
  • 21. Pivotal Container Service (PKS) Provides the control plane for provisioning and managing Kubo releases Joint development effort between Pivotal and VMWare Kubernetes Dial Tone: • Health management • Framework for Metrics and Logging • Autoscaling • Persistence interface • Networking interface Control Plane: • Provisioning Engine • Self-service Clusters • Software Update Automation • Load balancing • Networking • Persistence • Multi-tenancy
  • 22. PKS: Provisioning Engine 22 PKS Service Broker Release templates Manifest Kubo Release
  • 23. 23 PKS Service Broker Release templates Manifest Kubo Release create cluster (with upgrade policy) PKS: Self-service Clusters
  • 24. 24 PKS: Networking (different options available) BOSH network Worker Worker 10.0.30.12 ContainerContainerContainer 10.200.2.6C2C overlay 10.200.1.510.200.1.4 10.0.30.11 kube-proxy iptables kube-proxy iptables Service network Service
  • 25. BOSH GCP
 Service
 Broker Harbor NSX-T Kubernetes K8s Cluster K8s Cluster K8s Cluster Built with open-source Kubernetes — Constant compatibility with the current stable release of Kubernetes, operated by BOSH. No proprietary extensions. 
 Production-ready — Highly available from apps to infrastructure, no single points of failure. Built-in health checks, scaling, auto-healing and rolling upgrades. 
 Multicloud — BOSH provides a reliable and consistent operational experience. For any cloud. Network management and security out-of-the-box with VMware NSX-T. Multi-cloud, multi-hypervisor. GCP APIs access — The GCP Service Broker allows apps to transparently access Google Cloud APIs, from anywhere. Easily move workloads to/from Google Container Engine (GKE). 
 Fully automated Ops — Fully automated deploy, scale, patch, upgrade. No downtime. Use CD pipelines to deploy your platform, too. VMware GCP Azure Openstack AWS PKSController
  • 26. Apps are constantly compatible between PKS and Google Container Engine (GKE) Google Container Engine (GKE) Pivotal Container Service (PKS) Private and Public Cloud Public Cloud (on GCP)
  • 27. VMware PKS Analytics Automation SecurityOperations MonitoringLogging Physical Infrastructure Container Registry vSphere vSAN Kubernetes on BOSH (Kubo) NSX GCP Service 
 Broker masteretcd workermasteretcd worker
  • 28. Leveraging more than one abstraction Other Broker Services Platform Services Logging Metrics Monitoring Pivotal Application Service Application Application VMware GCP Azure Openstack AWS PKSController GCP
 Service
 Broker Harbor NSX-T Kubernetes K8s Cluster K8s Cluster K8s Cluster
  • 29. Unified Platform Experience Shared Logging and Metrics Shared Networking Shared Security
  • 31. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 31 BOSH GCP
 Service
 Broker Harbor NSX-T Kubernetes K8s Cluster K8s Cluster K8s Cluster Built with open-source Kubernetes — Constant compatibility with the current stable release of Kubernetes, operated by BOSH. No proprietary extensions. 
 Production-ready — Highly available from apps to infrastructure, no single points of failure. Built-in health checks, scaling, auto-healing and rolling upgrades. 
 Multicloud — BOSH provides a reliable and consistent operational experience. For any cloud. Network management and security out-of-the-box with VMware NSX-T. Multi-cloud, multi-hypervisor. GCP APIs access — The GCP Service Broker allows apps to transparently access Google Cloud APIs, from anywhere. Easily move workloads to/from Google Container Engine (GKE). 
 Fully automated Ops — Fully automated deploy, scale, patch, upgrade. No downtime. Use CD pipelines to deploy your platform, too. VMware GCP Azure Openstack AWS PKSController How does this all work? • Availability and Resiliency • Operational Efficiency • Multi-tenancy isolation • Security • Extensibility Guiding Principles
  • 32. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 32 BOSH PKSControlPlane Pivotal Container Service
  • 33. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ BOSH Pivotal Container Service PKSControlPlane Create cluster
  • 34. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 34 Availability Zone B Availability Zone A HA and Health Management - Kubelet watches and restart containers - Bosh director watches and restarts nodes - Bosh agent watches and restarts processes - Bosh distributes deployments across AZ’s Kubelet Kube-proxy Pod Pod K8s Node Pod API Server Kube Scheduler K8s Master Controller Manager Bosh agent Bosh agentBosh director Watches and restarts VMs Availability Zone A Availability Zone B Create cluster
  • 35. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 35 BOSH Pivotal Container ServicePKSControlPlane Kubelet Kube-proxy Fluentd Pod Pod Pod K8s Node Kubelet Kube-proxy Fluentd Pod Pod Pod K8s Node Kubelet Kube-proxy Pod Pod Pod K8s Worker API Server Kube Scheduler K8s Master Controller Manager Node Controller Replication Controller Endpoints Controller Service Accounts & Token Etcd GCP
 Service
 BrokerHarbor NSX-T Proxy NCP T1 Routers NSX-T T1 Routers Other components Included component, but optional usage CFCR / Kubo
  • 36. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 36 BOSH Pivotal Container ServicePKSControlPlane Kubelet Kube-proxy Fluentd Pod Pod Pod K8s Node Kubelet Kube-proxy Fluentd Pod Pod Pod K8s Node Kubelet Kube-proxy Pod Pod Pod K8s Worker API Server Kube Scheduler K8s Master Controller Manager Node Controller Replication Controller Endpoints Controller Service Accounts & Token Etcd GCP
 Service
 BrokerHarbor NSX-T Proxy NCP T1 Routers NSX-T T1 Routers Other components Included component, but optional usage CFCR / Kubo Kubernetes
  • 37. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Kubernetes components API Server Kube Scheduler K8s Master Controller Manager Etcd K-V Store Kubelet Kube-proxy K8s Worker Pod Pod Pod Kubelet Kube-proxy K8s Worker Pod Pod Pod Kubelet Kube-proxy K8s Worker Pod Pod Pod CNI Persistent Volume Virtual switch Network edge router Volume mount Aws, Azure, GCP, vSphere SCSI, FC, NFS, Photon, Scale IO, … Provisioner NSX-T, Flannel, Calico, GCE, Nuange, OVN, Kube-router… CNI Plugin Protobuf Network edge routerNetwork edge router
  • 38. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 38 API Server Kube Scheduler K8s Master Controller Manager Etcd K-V Store Kubelet Kube-proxy K8s Worker Pod Pod Pod Kubelet Kube-proxy K8s Worker Pod Pod Pod Kubelet Kube-proxy K8s Worker Pod Pod Pod CNI Persistent Volume Virtual switch Network edge router Volume mount Aws, Azure, GCP, vSphere SCSI, FC, NFS, Photon, Scale IO, … Provisioner NSX-T, Flannel, Calico, GCE, Nuage, OVN, Kube-router… CNI Plugin Protobuf Network edge routerNetwork edge router Pluggable Networking (CNI)
  • 39. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 39 BOSH Pivotal Container Service PKSController GCP
 Service
 Broker BOSH network 10.0.1.5 Harbor 10.0.1.2 Etcd 10.0.1.6 Etcd 10.0.1.6 API Server Kube Scheduler K8s Master Controller Manager Node Controller Replication Controller Endpoints Controller Service Accounts & Token 10.0.1.4 Kubelet Kube-proxy Fluentd Pod Pod Pod K8s Node 10.0.1.3 Kubelet Kube-proxy Fluentd Pod Pod Pod K8s Node 10.0.1.3 Pod Pod Pod K8s Worker Kubelet Kube-proxy 10.0.1.3 Networking
  • 40. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 40 BOSH PKSController GCP
 Service
 Broker Harbor Etcd API Server Kube Scheduler K8s Master Controller Manager Node Controller Replication Controller Endpoints Controller Service Accounts & Token Base Network 10.0.0.11 10.0.0.16 10.0.0.17 10.0.0.13 BOSH Network NSX-T Mgr NSX-T NCP .18 .19 10.0.0.21 10.0.0.22 Kubelet NSX Kube-Proxy OVS K8s Worker K8s Worker Kubelet
  • 41. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 41 BOSH PKSController GCP
 Service
 Broker Harbor Etcd API Server Kube Scheduler K8s Master Controller Manager Node Controller Replication Controller Endpoints Controller Service Accounts & Token Base Network 10.0.0.11 10.0.0.16 10.0.0.17 10.0.0.13 BOSH Network NSX-T Mgr NSX-T NCP .18 .19 10.0.0.21 10.0.0.22 Kubelet Pod Pod NSX Kube-Proxy OVS K8s Worker Pod Pod NSX Kube-Proxy K8s Worker K8K8s NameSpace A = NSX Logical Switch K8K8s NameSpace B = NSX Logical Switch T1 10.172.10.2 10.172.100.0/24 192.168.100.0/24 NO_NAT NAT T1 10.172.10.3 T0 10.172.10.1 Kubelet OVS NSX CNI PLUGIN Overlay Networks
  • 42. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 42 -T Pivotal Container Service Network Policy Management Multi-Tenancy Isolation Network Security Groups Dynamic Load Balancing & Ingress Comprehensive VM + Container security model
  • 43. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 43 kubectl apply -f <file.yml> Network Policies Management BOSH Pivotal Container Service PKSController NSX-T Proxy NCP T1 RoutersT1 Routers Other components NSX-T Kubelet Kube- proxy Fluentd Pod Pod Pod K8s Node Kubelet Kube- proxy Fluentd Pod Pod Pod K8s NodeNode NSX Router Network policy management API Server Kube Scheduler K8s Master Controller Manager Etcd new policy Apply Kubelet K. Proxy Pod Pod Pod
  • 44. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 44 Deployment Topologies & Multi-Tenancy Multi-cluster Single cluster K8s Cluster A K8s Cluster BOSH PKSController Namespace A Namespace B Namespace C BOSH PKSController K8s Cluster B K8s Cluster C
  • 45. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 45 admin@k8s-master:~$ kubectl create namespace foo namespace ”foo" created admin@k8s-master:~$ kubectl create namespace bar namespace ”bar" created admin@k8s-master:~$ kubectl run nginx-foo --image=nginx -n foo deployment "nginx-foo" created admin@k8s-master:~$ kubectl run nginx-bar --image=nginx -n bar deployment "nginx-bar" created Namespace: foo Namespace: bar NSX / K8s topology 10.24.0.0/24 10.24.1.0/24 10.24.2.0/24 NAT boundary NAT boundary K8s nodesK8s Masters NameSpace as a Tenancy Construct NSX-T Automatically Provides Each Namespace a secure network & routing
  • 46. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 46 admin@k8s-master:~$ kubectl label pods nginx-foo-3492604561-nltrf secgroup=web -n foo Pod "nginx-nsx-3492604561-nltrf" labeled admin@k8s-master:~$ kubectl label pods nginx-bar-2789337611-z09x2 secgroup=db -n bar pod "nginx-k8s-2789337611-z09x2" labeled admin@k8s-master:~$ kubectl get pods --all-namespaces -Lsecgroup NAMESPACE NAME READY STATUS RESTARTS AGE SECGROUP k8s nginx-foo-2789337611-z09x2 1/1 Running 0 58m web nsx nginx-bar-3492604561-nltrf 1/1 Running 0 1h db Namespace: foo Namespace: bar NSX / K8s topology 10.24.0.0/24 10.24.1.0/24 114.4.10.0/26 NAT boundary NAT boundary Web • Security Groups are defined in NSX with ingress and egress policy • Each Security Group could be micro-segmented to protect Pods from each other DB
  • 47. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 47 App / Container Deployment, Services & Routing cf push App myapp.mydomain.net Pivotal Cloud Foundry Application Runtime Pivotal Container Service kubectl run ???? ????
  • 48. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 48 App / Container Deployment, Services & Routing cf push App myapp.mydomain.net Pivotal Cloud Foundry Application Runtime Pivotal Container Service kubectl run ???? ????
  • 49. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 49 BOSH Pivotal Container Service PKSController Kubelet Kube-proxy Pod K8s Worker K8s Master Controller Manager Node Controller Replication Controller Endpoints Controller Service Accounts & Token kubectl run <image> Harbor Etcd Pod API Server Kube Scheduler Pod Pod
  • 50. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 50 ??? kubectl run <docker img> K8s Worker Kubelet Kube-proxy Pod Pod Pod 192.168.0.4 192.168.0.3 192.168.0.2 API Server Kube Scheduler K8s Master Controller Manager API Server Kube Scheduler K8s Master Controller Manager K8s Worker Kubelet Kube-proxy Pod Pod Pod 192.168.0.4 192.168.0.3 192.168.0.2 K8s Worker Kubelet Kube-proxy Pod Pod 192.168.0.4 192.168.0.3 Pod 192.168.0.2 K8s Cluster Base Network Overlay Network Cluster network (conceptual)
  • 51. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 51 kubectl run <docker img> K8s Worker Kubelet Kube-proxy Pod Pod Pod 192.168.0.4 192.168.0.3 192.168.0.2 Cluster network (conceptual) API Server Kube Scheduler K8s Master Controller Manager API Server Kube Scheduler K8s Master Controller Manager K8s Cluster K8s Worker Kubelet Kube-proxy Pod Pod Pod 192.168.0.4 192.168.0.3 192.168.0.2 Base Network Overlay Network Service K8s Worker Kubelet Kube-proxy Pod Pod 192.168.0.4 192.168.0.3 Pod 192.168.0.2
  • 52. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 52 NodePort K8s Worker Kubelet Kube-proxy Pod Pod Pod 192.168.0.4 192.168.0.3 192.168.0.2 Cluster network (conceptual) API Server Kube Scheduler K8s Master Controller Manager API Server Kube Scheduler K8s Master Controller Manager K8s Cluster K8s Worker Kubelet Kube-proxy Pod Pod Pod 192.168.0.4 192.168.0.3 192.168.0.2 Base Network Overlay Network kubectl expose deployment NodePort NodePort K8s Worker Kubelet Kube-proxy Pod Pod 192.168.0.4 192.168.0.3 Pod 192.168.0.2 host:port NodePort Load Balancer host
  • 53. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 53 NodePort K8s Worker Kubelet Kube-proxy Pod Pod Pod 192.168.0.4 192.168.0.3 192.168.0.2 API Server Kube Scheduler K8s Master Controller Manager API Server Kube Scheduler K8s Master Controller Manager K8s Worker Kubelet Kube-proxy Pod Pod Pod 192.168.0.4 192.168.0.3 192.168.0.2 kubectl expose deployment Load Balancer NodePort K8s Worker Kubelet Kube-proxy Pod Pod 192.168.0.4 192.168.0.3 Pod 192.168.0.2 NodePort Cloud Load Balancer host Dynamically provisioned Kubernetes Cluster Cloud 35.190.151.218 Public IP -T
  • 54. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 54 Services: Ingress An API object that manages external access to the services in a cluster, typically HTTP. Ingress can provide load balancing, SSL termination and name-based virtual hosting. K8s Worker Kubelet Kube-proxy PodPodPod K8s Worker Kubelet Kube-proxy PodPodPodS2 S1 K8s Worker Kubelet Kube-proxy PodPod Pod Ingress /foo /bar foo.bar.com Deploy
  • 55. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 5 BOSH Pivotal Container Service PKSController Kubelet Kube-proxy Fluentd Pod Pod K8s Node API Server Kube Scheduler K8s Master Controller Manager Node Controller Replication Controller Endpoints Controller Service Accounts & Token Pod 192.168.0.1 192.168.0.4 192.168.0.3 192.168.0.2 Pivotal Application Service Route-sync GO Router myapp.apps.myorg.net kubectl expose deployment (NodePort w/ route sync) NodePort Integration with CFAR Routing
  • 56. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 56 API Server Kube Scheduler K8s Master Controller Manager Etcd K-V Store Kubelet Kube-proxy K8s Worker Pod Pod Pod Kubelet Kube-proxy K8s Worker Pod Pod Pod Kubelet Kube-proxy K8s Worker Pod Pod Pod CNI Persistent Volume Virtual switch Network edge router Volume mount Aws, Azure, GCP, vSphere SCSI, FC, NFS, Photon, Scale IO, … Provisioner Protobuf Network edge routerNetwork edge router Pluggable Persistent Volumes
  • 57. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 57 Pod Container Container Host mount Fiber Channel, iSCSI, NFS GCP, Azure, AWS, vSphere Volumes and Persistent Volumes Legacy Systems Stateful Systems Database
  • 58. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 58 kubectl apply -f <file.yml> Pivotal Container Service PKSController K8s Master Etcd Worker Kubelet K. Proxy Pod Pod Worker Kubelet K. Proxy Pod Pod Worker Kubelet K. Proxy Pod Pod BOSH
  • 60. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 60 Stateful Sets Stable, unique network identifiers Stable, persistent storage 
 Ordered, graceful deployment and scaling 
 Ordered, automated rolling updates * stable = survives Pod rescheduling 
 Pod Stateful set Name = web Replicas = 3 Pod web-0 web-1 web-2 Pod PV-claim-web-0 Each Pod gets an unique ID and Persistent Volumes PV-claim-web-1 PV-claim-web-2
  • 61. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 61 Stateful Sets Apps / clusters using specific IPs / names to communicate Apps / clusters relying on persistent and durable storage on specific mount points Apps / clusters with specific starting order for components Stable, unique network identifiers Stable, persistent storage 
 Ordered, graceful deployment and scaling 
 Ordered, automated rolling updates * stable = survives Pod rescheduling 
 Clustered, stateful workloads or legacy apps
  • 62. Learn More. Stay Connected. 62 #springone@s1p