JAX London 2021: Jumpstart Your Cloud Native Development: An Overview of Practices and Tooling
0 likes162 views
The document discusses the importance of continuous delivery in cloud-native development, emphasizing the need for safety and speed throughout the engineering workflow. It highlights modern software trends like microservices and Kubernetes, and introduces various practices and tools such as artifact syncing, development environment bridging, and GitOps to enhance delivery efficiency. Ultimately, it stresses adopting best practices to mitigate challenges associated with transitioning to cloud-native services.
![@danielbryantuk
Continuous delivery of value
6
“Continuous delivery is achieved when stability and speed
can satisfy [your] business demand.
Discontinuous delivery occurs when stability and speed
are insu
ff
icient.”
- Steve Smith (@SteveSmith_Tech)](https://image.slidesharecdn.com/jaxlondon-jumpstartyourcloudnativedevelopment-anoverviewofpracticesandtooling-211005125725/85/JAX-London-2021-Jumpstart-Your-Cloud-Native-Development-An-Overview-of-Practices-and-Tooling-6-320.jpg)
![@danielbryantuk
Ska
ff
old: “Source to K8s” CLI Tool
$ skaffold dev
19
Listing files to watch...
- skaffold-example
Generating tags...
- skaffold-example -> skaffold-example:v1.1.0-113-g4649f2c16
Checking cache...
- skaffold-example: Not found. Building
Found [docker-desktop] context, using local docker daemon.
Building [skaffold-example]...
Sending build context to Docker daemon 3.072kB
Step 1/6 : FROM golang:1.12.9-alpine3.10 as builder
---> e0d646523991
Step 2/6 : COPY main.go .
---> Using cache
---> e4788ffa88e7
Step 3/6 : RUN go build -o /app main.go
---> Using cache
---> 686396d9e9cc
Step 4/6 : FROM alpine:3.10
---> 965ea09ff2eb
Step 5/6 : CMD ["./app"]
---> Using cache
---> be0603b9d79e
Step 6/6 : COPY --from=builder /app .
---> Using cache
---> c827aa5a4b12
Successfully built c827aa5a4b12
Successfully tagged skaffold-example:v1.1.0-113-g4649f2c16
Tags used in deployment:
- skaffold-example -> skaffold-example:c827aa5a4b12e707163842b803d666eda11b8ec20c7a480198960cfdcb251042
local images can't be referenced by digest. They are tagged and referenced by a unique ID instead
Starting deploy...
- pod/getting-started created
Watching for changes...
[getting-started] Hello world!
[getting-started] Hello world!
[getting-started] Hello world!](https://image.slidesharecdn.com/jaxlondon-jumpstartyourcloudnativedevelopment-anoverviewofpracticesandtooling-211005125725/85/JAX-London-2021-Jumpstart-Your-Cloud-Native-Development-An-Overview-of-Practices-and-Tooling-19-320.jpg)
![@danielbryantuk
Telepresence: Local-to-Remote bridge
$ telepresence intercept dataprocessingnodeservice --port 3000
26
Using deployment dataprocessingnodeservice
intercepted
State : ACTIVE
Destination : 127.0.0.1:3000
Intercepting: HTTP requests that match all of:
header("x-telepresence-intercept-id") ~= regexp ("76a1e848-1829-74x-1138-e3294c1e9119:dataprocessingnodeservice")
Preview URL : https://[random-subdomain].preview.edgestack.me
www.youtube.com/watch?v=W_a3aErN3NU](https://image.slidesharecdn.com/jaxlondon-jumpstartyourcloudnativedevelopment-anoverviewofpracticesandtooling-211005125725/85/JAX-London-2021-Jumpstart-Your-Cloud-Native-Development-An-Overview-of-Practices-and-Tooling-26-320.jpg)
JAX London 2021: Jumpstart Your Cloud Native Development: An Overview of Practices and Tooling
- 1. Jumpstart Your Cloud Native Development: An Overview of Practices and Tooling Daniel Bryant
- 2. @danielbryantuk tl;dr 2 • Continuous delivery requires safety and speed throughout the engineering workflow • Bringing old mental models and tools can increase toil when building, testing, and running cloud native services • Artifact syncing, dev environment bridging, and GitOps increase both safety and speed
- 4. @danielbryantuk Continuous delivery of value 4 “Continuous delivery is achieved when stability and speed can satisfy business demand. Discontinuous delivery occurs when stability and speed are insu ff icient.” - Steve Smith (@SteveSmith_Tech)
- 5. @danielbryantuk Continuous delivery of value 5 “Continuous delivery is achieved when stability and speed can satisfy business demand. Discontinuous delivery occurs when stability and speed are insu ff icient.” - Steve Smith (@SteveSmith_Tech)
- 6. @danielbryantuk Continuous delivery of value 6 “Continuous delivery is achieved when stability and speed can satisfy [your] business demand. Discontinuous delivery occurs when stability and speed are insu ff icient.” - Steve Smith (@SteveSmith_Tech)
- 7. 7 Two modern so ft ware trends for speed and safety
- 8. @danielbryantuk Trend #1: Microservices 8 • Modularisation • Aka “microservices” • Independently: • Build • Release • Scale • Operate • Strive for • High cohesion • Loose coupling
- 9. @danielbryantuk Trend #2: Kubernetes 9 • Automated orchestration of container-based services • Declarative config (with reconciliation loops) • Common API/platform to support and extend • But… a non-trivial set of abstractions/components for developers • And it’s not a PaaS
- 10. @danielbryantuk tl;dr 10 With cloud native, the focus is o ft en on speed
- 11. The Kubernetes Migration Journey(s) 11 Exploration Organizational buy-in Cloud-native organization (Loosely coupled, highly aligned) Building services, but we’re not actually moving faster Let’s release our microservices as a monolith Centres of excellence (Tech & process changes) No coordination results in disasters and blame (deploy and operate K8s same as old infra) Investing in “paved road” platforms And new dev workflows Clinging to old mental models and “the way it’s always been done”
- 12. 12 Safety The ability to quickly ship changes to production, without fear
- 13. Safety requires multiple strategies 13 Prevention Mitigation Detection Response Most common strategy (e.g., testing, QA, gated release) Critical strategies for cloud-native organizations when the application is a “complex adaptive system” } }
- 14. Safety Strategies in the Cloud 14 Prevention Mitigation Detection Response • Testing: integration tests, user acceptance tests, … • High-fidelity replica environments for dev & test • Engineering for resilience • Progressive delivery: canary release, blue/green rollouts, … • Observability • Chaos engineering • Instant rollback & traceability • Runbooks, fast dev / test loops • Blameless postmortems • Game days
- 15. Safety needs to be part of your workflow. 15 Develop Test Deploy/ Release Run Progressive Delivery Rollback Observability Chaos Engineering Runbooks Fast dev loop Fast feedback Realistic test environments
- 16. 16 Develop Test Deploy/ Release Run Progressive Delivery Rollback Observability Chaos Engineering Runbooks Fast dev loop Fast feedback Realistic test environments Developer Self-Service
- 17. @danielbryantuk 17 Patterns, practices, and tooling
- 18. @danielbryantuk Pattern: Artifact Syncing 18 • Pain points: • Can’t run all required dependent services locally • Sick of slow dev loop: code, build image, push to remote registry • Integration tests rely on mocks (with implicit assumptions) • Solution • Deploy all services to remote environment and sync local changes (combine with buildpacks) • O ft en combined with development environment bridging • Example tool • Ska ff old (ksync, Tilt, Garden)
- 19. @danielbryantuk Ska ff old: “Source to K8s” CLI Tool $ skaffold dev 19 Listing files to watch... - skaffold-example Generating tags... - skaffold-example -> skaffold-example:v1.1.0-113-g4649f2c16 Checking cache... - skaffold-example: Not found. Building Found [docker-desktop] context, using local docker daemon. Building [skaffold-example]... Sending build context to Docker daemon 3.072kB Step 1/6 : FROM golang:1.12.9-alpine3.10 as builder ---> e0d646523991 Step 2/6 : COPY main.go . ---> Using cache ---> e4788ffa88e7 Step 3/6 : RUN go build -o /app main.go ---> Using cache ---> 686396d9e9cc Step 4/6 : FROM alpine:3.10 ---> 965ea09ff2eb Step 5/6 : CMD ["./app"] ---> Using cache ---> be0603b9d79e Step 6/6 : COPY --from=builder /app . ---> Using cache ---> c827aa5a4b12 Successfully built c827aa5a4b12 Successfully tagged skaffold-example:v1.1.0-113-g4649f2c16 Tags used in deployment: - skaffold-example -> skaffold-example:c827aa5a4b12e707163842b803d666eda11b8ec20c7a480198960cfdcb251042 local images can't be referenced by digest. They are tagged and referenced by a unique ID instead Starting deploy... - pod/getting-started created Watching for changes... [getting-started] Hello world! [getting-started] Hello world! [getting-started] Hello world!
- 20. @danielbryantuk Ska ff old: “Source to K8s” CLI Tool 20 getambassador.io/summer-of-k8s/code/week3
- 21. @danielbryantuk Ska ff old: “Source to K8s” CLI tool 21 skaffold dev & skaffold debug • Detects changes in your source code and handles the pipeline to build, push etc skaffold build & skaffold deploy • Execute as part of your CI/CD pipeline, or ska ff old run end-to-end skaffold render • Build your images and render templated Kubernetes manifests for use in GitOps workflows
- 22. @danielbryantuk Pattern: Dev Environment Bridging/Extension 22 • Pain points: • Can’t run all required dependent services locally • Integration tests rely on mocks (with implicit assumptions) • Need “hot reload” coupled to remote services/resources • Solution • Deploy all services to remote environment and proxy/re-route tra ff ic to/from a local running copy of a service (or subset of services) • Example tool • Telepresence (kubefwd, kubectl port-forward)
- 23. @danielbryantuk Telepresence: Local-to-Remote bridge $ telepresence intercept dataprocessingnodeservice --port 3000 23 getambassador.io/summer-of-k8s/code/week3
- 25. @danielbryantuk Telepresence: Local-to-Remote bridge 25 telepresence connect • Open a tunnel to the remote cluster; exposes “in-cluster” services/network/DNS telepresence intercept my-service —port 3000 • Re-routes (intercepts) tra ff ic to my-service in the remote cluster to my local machine telepresence login & telepresence intercept • Create preview URL to isolate and share results of the intercept
- 26. @danielbryantuk Telepresence: Local-to-Remote bridge $ telepresence intercept dataprocessingnodeservice --port 3000 26 Using deployment dataprocessingnodeservice intercepted State : ACTIVE Destination : 127.0.0.1:3000 Intercepting: HTTP requests that match all of: header("x-telepresence-intercept-id") ~= regexp ("76a1e848-1829-74x-1138-e3294c1e9119:dataprocessingnodeservice") Preview URL : https://[random-subdomain].preview.edgestack.me www.youtube.com/watch?v=W_a3aErN3NU
- 27. @danielbryantuk Pattern: GitOps 27 • Pain points: • No repeatable, auditable, & understandable deployment of infra and apps • Slow hand-o ff s and manual deploys • Solution • Use version control (git) as single source of truth • Declarative config; reconciled against target cluster • Example tool • Argo CD (Flux, JenkinsX, werf) https://www.weave.works/blog/what-is-gitops-really
- 28. @danielbryantuk Argo CD: GitOps for K8s 28 $ argocd app create guestbook --repo https://github.com/argoproj/argocd-example-apps.git / --path guestbook --dest-server https://kubernetes.default.svc --dest-namespace default getambassador.io/summer-of-k8s/ship/week2
- 29. @danielbryantuk GitOps in a Nutshell 29 https://www.weave.works/blog/the-gitops-pipeline
- 30. @danielbryantuk Argo Rollouts: K8s Progressive Delivery Controller 30 getambassador.io/summer-of-k8s/ship/week3
- 31. @danielbryantuk Putting it All Together: Safety and Speed 31 https://www.weave.works/blog/what-is-gitops-really Development Production Artifact syncing, Dev Env Bridging GitOps Speed Safety
- 32. @danielbryantuk 32 What about the platform and the people?
- 33. @danielbryantuk Workflow: Full Cycle Development 33 • App teams have full responsibility (and authority) for delivering a service, and ultimately, value to users • Increase agility by accelerating the feedback loop • https://netflixtechblog.com/full-cycle-developers-at-netflix-a08c31f83249
- 34. @danielbryantuk Conclusion 34 • Continuous delivery of value to users requires a focus on safety and speed throughout the engineering workflow • Bringing old mental models and tools can add toil to the process of building, deploying, and operating cloud native services • Adopt best practices first, such as artifact syncing, dev environment bridging, and GitOps to increase both safety and speed
- 35. @danielbryantuk • Learn more: • www.getambassador.io/resources • www.getambassador.io/use-case/local-kubernetes-development • www.infoq.com/profile/Daniel-Bryant • Find me in: • Ambassador OSS Slack: a8r.io/slack • Twitter @danielbryantuk Many thanks! 35