Rancher master class globalized edge workloads with k3s
- 1. Globalized Edge Workloads with K3s August 13, 2019 Joseph D. Marhee of Packet Labs
- 3. MISSION At Packet Labs, we craft solutions to uncommon problems that live at the intersection of hardware, software and networks.
- 4. Meet Joseph D. Marhee Ecosystem Engineer @ Packet Labs â—Ź Customer-first solutions obsessive. â—Ź Long time IaaS systems solutions engineer. â—Ź Has worked with Kubernetes in the provider space since 2015 and has a lot of opinions about it. â—Ź @joseph.packet on the Packet Community Slack
- 5. Labs Team My D. Truong Senior Director Scott Anderson VP, Outreach Ed Vielmetti Special Projects Director Joseph Marhee Ecosystem Engineer Carl Perry Ecosystem Engineer Scott Burns Senior Director, R & D ➔49 contributing team members on Github ➔15,000 users around the world
- 6. Programming
- 7. Overview Why Edge Compute? Why K3s? Why Bare metal? Why anything? â—Ź Infrastructure Edge, Mobile Edge â—Ź Edge Architectures â—Ź K3s Powered Applications & Emerging Uses â—Ź Orchestration â—Ź Containers on Bare Metal â—Ź How does Packet enhance these patterns? â—Ź Demonstration of Packet K3s
- 8. Quick Recap of Traditional K8s Architecture Concepts “The network is the computer” ● Kubernetes is, conceptually, a scheduler and a reconciliation loop. ● APIServer takes instructions to scheduler ● Scheduler informs node agent (Kubelet) ● Kubelet reports to control plane on state, metrics, provision success/failure. ● Define -> Deploy > loop(Check, Act)
- 9. What’s the Infrastructure Edge? ● Globally-distributed points of presence of, for example, an IaaS endpoint. ● These can be local to where IoT edge devices are located. ● Architecturally, this is a simple concept, but in practice, the tools may not be intuitive, easily globalized and standardized.
- 10. Edge Architecture Example: Replicated workloads - Database engines with global replication support (think something like FaunaDB) - Ingests and validates data on one instance, shards across other clusters globally in a mesh of facility clusters - The pattern would not be unlike a “Kubernetes Underlay” in the absence of robust federation.
- 11. Edge Architecture Example: FaaS Users are load-balanced between sites, in this example, and when a request comes into the LB, a facility is selected. This pattern is ideal for FaaS-based workloads: - Tools like Fission.io or OpenFaaS on each endpoint - Can be a local POP for representing-- at the edge- data store in your core site persistent storage. - Apply this to mobile edge to infrastructure edge patterns.
- 12. Practical Use Case: Content Delivery Source If familiar with a traditional Content Delivery Network (CDN) model, this is one such pragmatic use case for K3s: - Core site running a full cluster - Edge sites with disposable deployments, rapid re- deployment, serving a cached data.
- 13. Orchestration The ability to roll out and manage a fleet of high number of comparatively low- powered on-prem/mobile devices, and more robust instances in a local facility is an important one for edge compute. Traditional configuration management aside, tools like: - Terraform (only as opinionated as your provider/platform, on-prem or in DC) - Wrappers like Gitops - Image builders like Packer Can speed up CI/CD of these resources to your edge sites (mobile and infrastructure).
- 14. Why Containers on Bare Metal? “Stop killing kittens and melting the ice caps: Run containers on bare metal already.” - Bryan Cantrill Virtualization can be an inefficient* abstraction for edge architectures: ● Low-cost hardware for small deployments available (removes the costly virtualization layer-- particularly at the edge) ● Container security less of a question (see: the Unikernel v. Container-in-VM debate circa 2015) ● For edge: maximizing benefits of the hardware platform features for single-tenancy. ● K3s enables common patterns for CI/CD, deployment, and rapid architecture rollout. * if you are not in the business of providing VMs, and if you are only servicing containers for your entire environment
- 15. How does Packet enable Kubernetes? â—Ź The speed and developer friendliness of cloud provider tools. â—Ź Unopinionated platform that enables: â—‹ Compatibility with nearly all community solutions â—‹ Partners to target Packet effectively â—Ź Network automation suitable for: â—‹ LoadBalancer type Services â—‹ Cluster networking (and between clusters and facilites) â—‹ Location-aware routing with Anycast IP addresses
- 16. Packet and the Edge â—Ź Packet offers multiple core sites and edge locations-- a point of presence in most areas. â—Ź Layer 2 & 3 network automation, Anycast IP addressing, and diverse NICs across a broad set of specs means that these features can be leveraged for K3s. â—Ź Network volume storage in core sites, attached storage on all host types. â—Ź Building an Infrastructure Edge-facing architecture becomes highly declarative using common tooling like Terraform, Ansible, Packer, to automate provisioning, and continuous integration of an edge site.
- 17. Packet K3s Demo Deploying a global, location-aware application on K3s. Using orchestration and provisioning tooling to manage a standardized cluster pool. Technologies used: - Anycast IPs/Global IPv4 - K3s - Packet Terraform provider to provision hosts, network, and bootstrap cluster.
- 18. I want to believe! What’s next? Running a K3s-powered edge strategy on Packet. Packet Labs Projects for Kubernetes in Core and Edge site models on Packet: ● Kubernetes w/BGP ● Multi-architecture Kubernetes ● Packet K3s Some of the tools and software that can target an edge strategy discussed today: ● Fission.io ● OpenFaaS ● chick-fil-a/gitops ● FaunaDB
- 19. Conclusion â—Ź Any questions? â—Ź Want to get involved with improving the Kubernetes & K3s experiences on Packet?