SlideShare a Scribd company logo
Shipping code like a
@keptnProject
https://keptn.sh
https://github.com/keptn
Rob.Jahn@Dynatrace.com
Technical Partner Manager
& Keptn Advocate
2
Every CIOs vision of their Cloud Native investment
3
The unfortunate reality for most….
1. Complexity
2. Manual operations
3. Lack information
4. People
5. Identifying root cause
4* based on our ACM survey
5
Maturity Questions
Mean Time to Innovation (MTTI)
1. Are you on two week sprints?​
2. Do you also release 25 times a year?​
3. How long does it take you to deploy a new release to production?​
4. Do you measure feature adoption?
Mean Time to Remediate (MTTR)
1. How long does it take to rollback/roll forward a faulty release?​
2. To release any software in your organization, how many manual steps does it
take?​
6
MTTI
Mean Time to Innovation
MTTR
Mean Time to Remediate
4.8 days
4 hours
~ 10min
12.5 days 2 days ~ 1 hour
The reality and evidence supports
it!
https://dynatrace.ai/acsurvey
Only < 5% is „Cloud Native“
7
That is why we are building
Because cloud native delivery and operations is a BIG challenge for enterprises!
Cloud Native
8
.
https://dzone.com/articles/keep-calms-and-devops-c-is-for-culture
Culture first, but change is not easy…
9
Common Pipeline delivery challenges
1. Pipelines are semi-automated
2. Each team building duplicate delivery & integration mechanisms
3. Changing tooling causes a ton pipeline re-work
4. Lack software architecture for DevOps automation
5. Lack auditing support for customized tool integrations
6. Lack of standards
7. Too many technologies to keep up with
8. What else ????
10
So what is this thing?
• Open source initiative to make continuous delivery and operations automation
a commodity for modern cloud-native applications
• Enterprise-grade event-based framework for shipping and operating cloud-
native applications built to run on any Kubernetes cluster
• With Keptn ...
• Developers can focus on code instead of creating YAML files
• DevOps can focus on tools instead of building pipelines
• SREs can enforce processes instead of debugging problems
11
Automated OperationsContinuous Deployment
Core Capabilities -- accelerates building autonomous clouds
Event-drivenrunbook
automation
Productionproblemscan beautomatically
remediatedin real-timebyexecuting
runbooksthatrequirenomanual
intervention.
Self-healing blue/green
deployments
Deploymentsthatfollowthe“Operationsas
Code”paradigmautomaticallyremediate
problemsandgetyourdeploymentpipeline
workingagainin underaminute.
Automatedmulti-stage
unbreakable pipelines
GitOps-enableddeliverypipelineswith
automatedqualitygatessupportautomated
testingandmonitoring-as-a-service.
12
First: solves the Continuous Delivery Problem!
Many approaches today are brittle, error-prone and hard to maintain
13
Quote: “We spend more time in manual communicaton than remediating issues“
Second: has Continuous Operations at its Core!
ENGAGE TRIAGE FIND & ASSEMBLE RESOLVE RESTORE
MANUAL
COMMUNICATION
MANUAL
COMM
Before
After
RESTORERESOLVE
NUMBER
OF ISSUES BEFORE: mostly manual
AFTER: mostly automated
14
Increase Quality &
Level of Automation
Increase Speed &
Reduce Costs
Automated
Testing
Continuous
Performance
Auto Quality
Gates
AUTOMATE
OPERATIONS
AUTOMATE
DEPLOYMENT
AUTOMATE
MONITORING
Feature
Flagging
Adaptive
Scaling
Auto
Roll-Back
Canary
Releases
Blue /
Green
Deploymen
ts
Auto-
Remediation
AUTOMATE
QUALITY
Strategically Used as Pipeline Feature
Building Blocks for ACM/Cloud Natives!
15
Design Principals – follows patterns for modern applications
GitOps-based
collaboration
Operator patterns for all
logic components
Monitoring and
operations ascode
Built onandfor Kubernetes Event-drivenandserverless Pluggabletooling
16
Keptn architecture
Platform
keptn
cli
api eventbroker
REST
API
bridge datastore
gitconfiguration-service
internal-
keptn-
services
distributor
distributor
keptn services
Continuous
Delivery
Container
Registry
Operations
Automation AIOps
GitOps
Test
Automation ChatOps
Data
Provider
dev staging production keptnNamespaces:
distributor
Mini-kube AWS EKS Azure AKS Google GKE Openshift Pivotal PKS
17
Keptn Use Cases - Installation
1. One-Line Installation: on most popular k8s platforms
2. Zero-Touch Toolchain Integration: No custom tool integrations needed
3. Zero-Touch Cloud Native Services: Enables GitOps event-driven CD/CO for your
services
18
Config ChatOps IT Autom
Deploy Test Observe
One-Line Installation: $ keptn install --platform=[aks, gke, openshift]
1. Keptn installer pod
2. Keptn pods and services(such as logging, keptn bridge,event broker)
3. Keptn routes (such asauthenticator, bridge, gate-keeper,helm service)
4. Keptn channels (such as configuration-changed, deployment-finished, evaluation-done)
5. Istio and knative pods
19
Config ChatOps IT Autom
Deploy Test Observe
Zero-Touch Toolchain Integration: $ keptn wear uniform uniform.yaml
uniform.yaml
services:
- name: "dockerhub"
- name: "jenkins-service"
subscribedchannels:
- new-artifact
- deployment-finished
-name: "slack-service”
20
Zero-Touch Toolchain Integration:
Config ChatOps IT Autom
Deploy Test Observe
Pluggable tooling
All tools leveraged by keptn
can be replaced based
on your tool preferences.
Keptn Orders Demo -- tool chain ”uniform”
Config ChatOps IT Autom
Deploy Test Observe
• Configuration:GitHub
• Deploy:Helm
• Chat:Slack& MS Teams
• IT Automation:None
• Test:Jmeter
• Observe:PitometerQualityGates
Keptn Orders Demo Setup
DockerRegistrywithpre-builtimages
GitHub Org
Staging Production
Update GitHub repositories
- front-end
- catalog-service
- customer-service
- catalog-service
Dev
https://github.com/keptn-orders
23
Keptn Use Cases – Continuous Delivery
1. Re-Think Pipelines: Gone are the days of custom pipeline coding!
2. Automated Multi-Stage Delivery: Risk-Free auto deployment through multi-stage
delivery pipelines
3. Automated Quality Gates: stops bad changes before production using Pitometer
4. Self-Healing Blue/Green Deployments: reverts bad changes before impacting end-
users
5. Zero-Touch Toolchain Updates: Add/Remove/Replace tools without custom coding
6. Mastering Continuous Delivery: Risk-Free Automated Deployments
24
Config ChatOps IT Autom
Deploy Test Observe
Re-Think Pipelines: $ keptn create project keptn-sample shipyard.yaml
S
T
A
G
I
N
G
P
R
O
D
DirectUpdateC D
Blue/GreenUpdateC D
25
Config ChatOps IT Autom
Deploy Test Observe
Zero-Touch Cloud Native Services: $ keptn onboard service myservice [values.yaml]
S
T
A
G
I
N
G
P
R
O
D
DirectUpdateC D
Blue/GreenUpdateC D
PLACEHOLDER
PLACEHOLDER
As a developer, I want to safely deploy a new
version by just providing my docker image
27
Config ChatOps IT Autom
Deploy Test Observe
Automated Multi-Stage Delivery: $ keptn new artifact myservice:1.0.0
S
T
A
G
I
N
G
P
R
O
D
ScoreDirect PerformanceUpdate Promote?C D T O
ScoreBlue/GreenUpdate Keep?C D T O
PLACEHOLDER
PLACEHOLDER
1.0.0
1 1 90
/
100
1.0.0
1 1 1 75
/
100
P
R
O
M
O
T
E
K
E
E
P
28
Config ChatOps IT Autom
Deploy Test Observe
Automated Quality Gates: $ keptn new artifact myservice:2.0.0
S
T
A
G
I
N
G
P
R
O
D
ScoreDirect PerformanceUpdate Promote?C D T O
ScoreBlue/GreenUpdate Keep?C D T O
1.0.0
1 1 45
/
100
1.0.0
1 1 1
2.0.0
2 2
A
B
O
R
T
Pitometer ???
devices used to measure a ship's speed
relative to the water. They are used on
both surface ships and submarines.
Data from the pitometer log is usually
fed directly into the ship's navigation
system. (Wikipedia)
30
A Quick word on Keptn Quality Gates: Automated Deployment Validation
Metric Source &
Query
Grading Details
& Metric Score
Pitometer Specfile
Total Scoring
Objectives
2GB
Allocated Bytes (from Prometheus)
> 2GB: 0 Points
< 2GB: 20 Points
5%
2% < 2%: 0 Points
< 5%: 10 Points
> 5%: 20 Points
Conversion Rate (Dynatrace)
GraderSource
If value: 3GB
Score: 0
If value: 3.9%
Score: 10
Total Score: 10
31
Automated Quality Gates with Keptn’s Pitometer
#1 Call Pitometer Library
Pass / Fail
Promote?
Quality Gates as Code
•
First extensions by
https://www.dynatrace.com/news/blog/automated-deployment-and-architectural-validation-with-pitometer-and-keptn/
32
Pitometer Client
 Evaluates & Scores results
 Community can build more
 1 to many modules as required
 Queries data for requested
indicators
 Community can build more
Pitometer Module  Core library for Data, sources,
graders and perfspec file parsing
PerfSpec JSON
Grader Module
Graders
Graders
Data Module(s)
Custom Code
 Logic to assemble and configure
modules. Read file and pass to
Pitometer functions
Pitometer JSON Results
Keptn Pitometer: Extensible design using NodeJS Modules
https://github.com/keptn/pitometer
33
Quality Gate Examples and Metrics
Check 1
• Is bad coding
leading to higher
costs?
Check 2
• New dependencies? On
Purpose?
• Services connecting
accurately?
• Number of container
instances needed?
Check 3
• Are we jeopardizing
our SLAs?
• Does load balancing
work?
• Difference between
Canaries?
Check 4
• Did we introduce
new “hidden”
exceptions?
Metrics
 Memory usage
 Bytes sent /
received
 Overall CPU
 CPU per
transaction type
Metrics
 Number of incoming /
outgoing
dependencies
 Number of instances
running on containers
 Metrics
 Response Time
(Percentiles)
 Throughput & Perf
per Instance /
Canary
 Metrics
 Total Exceptions
 Exceptions by
Class & Service
34
Config ChatOps IT Autom
Deploy Test Observe
Self-Healing Blue/Green Deployments: $ keptn new artifact myservice:3.0.0
S
T
A
G
I
N
G
P
R
O
D
ScoreDirect PerformanceUpdate Promote?C D T O
ScoreBlue/GreenUpdate Keep?C D T O
1.0.0
1 1 85
/
100
1.0.0
1 1 1
2.0.0
2 2
3.0.0
3 3
3.0.0
3 3 3
80
/
100
P
R
O
M
O
T
E
R
E
V
E
R
T
As a DevOps Engineer, I want switch tools
without writing custom pipeline code
36
Config ChatOps IT Autom
Deploy Test Observe
Zero-Touch Toolchain Updates: $ keptn update uniform <+neo,+end2end,+spinnaker>
S
T
A
G
I
N
G
P
R
O
D
ScoreDirect PerformanceUpdate Promote?C D T O
ScoreBlue/GreenUpdate Keep?C D O
1.0.0
1 1
1.0.0
1 1 1
2.0.0
2 2
3.0.0
3 3
End2EndT
37
Config ChatOps IT Autom
Deploy Test Observe
Mastering Continuous Delivery: $ keptn new artifact myservice:4.0.0
S
T
A
G
I
N
G
P
R
O
D
ScoreDirect PerformanceUpdate Promote?C D T O
ScoreBlue/GreenUpdate Keep?C D O
1.0.0
1 1
1.0.0
1 1 1
2.0.0
2 2
3.0.0
3 3
4 4 4
End2EndT
4.0.0
4.0.0
4 4 95
/
100
90
/
100
K
E
E
P
P
R
O
M
O
T
E
38
Keptn Use Cases – Continuous Operations
1. Self-Healing Production: Automated Problem Remediation (Scale-Up, Scale-
Down, ...)
2. Self-Healing/Continuous BizDevOps: Automated Business Operations
Optimization, Turn on/off feature flags based on conversion rates ...
3. Auto-Protect Production & Business: Stop DDOS attacks, Redirect Bot Traffic ...
4. Chaos-Driven Operation Readiness: Chaos Engineering to validate your
production self-healing
As an SRE, I want production to auto-
remediate problems detected by monitioring
40
4 4
Self-Healing Production: $ keptn new problem <Services, Root Cause>
P
R
O
D
Evaluate
Decide
Act
Notify
4 Escalate
4 4
#1 Evaluate Problem #3 Execute Action #4 Notify on Success #5 Escalate on Failure#2 Decide on Remediation
41
• https://github.com/keptn/community
• Twitter - @keptnproject
• Slack - keptn.slack.com
• Biweekly community meetings
• Zoom https://dynatrace.zoom.us/j/578828410
• Notes https://tinyurl.com/keptn-meeting-notes
• Blogs– http://medium.com/keptn
• Web – http://keptn.sh
• Email- keptn@dynatrace.com
Jointhe community
Rob.Jahn@Dynatrace.com
Technical PartnerManager
& KeptnAdvocate
42
43
Keptn Orders Application
• Order processing application
• Web UI with 3 java spring boot microservices
with embedded databases
• Components pre-compiled, built as a Docker
image, and staged on dockerhub.com
• All source code can be found at
https://github.com/keptn-orders
44
Keptn-Orders GitHub Repo Structure
App Repos
front-end
customer-
service
order-service
catalog-
service
/jmeter/
basiccheck.jmx
front-end_load.jmx
/perfspec/
perfspec.json
Other files
* Source Code
* CI Build pipeline files
* Dockerfile
* Build scripts
Same structure as
front-end
Same structure as
front-end
Same structure as
front-end
orders-
project
Dev Branch
/Helm-Charts/
chart values files
/templates/
istio virtual gateway YAML
service YAML
deployment YAML
environment gateway YAML
Staging Branch
Same structure and files as Dev Branch
Production Branch
Same structure and files as Dev Branch
Folders &Files Keptn Repo Folders &Files
https://github.com/jahn-keptn-orders-aks2
45
VM Worker Pool
NAME SPACES
Kubernetes Demo Cluster Setup with Keptn & Deployed Keptn-Orders Application
Node 1
Dynatrace
Keptn
Staging
Production front-end
front-end
customer-
service
order-service
catalog-
service
bridge
OneAgent
Operator
OneAgent
(node 1)
front-end
customer-
service
order-service
catalog-
service
gatekeeper
Jmeter-
service
helm-service …
Dev front-end
customer-
service
order-service
catalog-
service
Node X

More Related Content

PPTX
Jenkins Online Meetup - Automated SLI based Build Validation with Keptn
PDF
2020.02.06 우리는 왜 glue를 버렸나?
PDF
AWS Personalize 중심으로 살펴본 추천 시스템 원리와 구축
PDF
Amazon S3を中心とするデータ分析のベストプラクティス
PDF
Writing Continuous Applications with Structured Streaming in PySpark
PDF
Experimentation to Industrialization: Implementing MLOps
PDF
ISMS 개발보안, 시큐어코딩을 위한 sonarqube의 활용.건국대학교병원.이제관 기술사
PDF
Elasticsearch
Jenkins Online Meetup - Automated SLI based Build Validation with Keptn
2020.02.06 우리는 왜 glue를 버렸나?
AWS Personalize 중심으로 살펴본 추천 시스템 원리와 구축
Amazon S3を中心とするデータ分析のベストプラクティス
Writing Continuous Applications with Structured Streaming in PySpark
Experimentation to Industrialization: Implementing MLOps
ISMS 개발보안, 시큐어코딩을 위한 sonarqube의 활용.건국대학교병원.이제관 기술사
Elasticsearch

What's hot (20)

PDF
gRPC と nginx による HTTP/2 サービスメッシュ構築
PDF
AWS Black Belt Tech シリーズ 2016 - Amazon SQS / Amazon SNS
PDF
CDC Stream Processing with Apache Flink
PDF
AWS와 함께 하는 클라우드 컴퓨팅 - 홍민우 AWS 매니저
PDF
MQTTとAMQPと.NET
PDF
[GKE & Spanner 勉強会] Cloud Spanner の技術概要
PPTX
오픈소스를 활용한 Batch_처리_플랫폼_공유
PDF
SBS 콘텐츠허브의 AWS 도입 사례 :: SBS 콘텐츠허브 :: AWS Media Day 2016
PDF
ドメイン駆動設計サンプルコードの徹底解説
PDF
20200212 AWS Black Belt Online Seminar AWS Systems Manager
PPTX
Mongo sharding
PDF
Amazon DocumentDB vs MongoDB 의 내부 아키텍쳐 와 장단점 비교
PDF
Introducing Change Data Capture with Debezium
PPTX
검색엔진이 데이터를 다루는 법 김종민
PPTX
Cluster API によるKubernetes環境のライフサイクル管理とマルチクラウド環境での適用
PDF
Kafka as your Data Lake - is it Feasible?
PDF
Storing 16 Bytes at Scale
PDF
JupyterHub: Learning at Scale
PDF
[236] 카카오의데이터파이프라인 윤도영
PPTX
C* Summit 2013: Eventual Consistency != Hopeful Consistency by Christos Kalan...
gRPC と nginx による HTTP/2 サービスメッシュ構築
AWS Black Belt Tech シリーズ 2016 - Amazon SQS / Amazon SNS
CDC Stream Processing with Apache Flink
AWS와 함께 하는 클라우드 컴퓨팅 - 홍민우 AWS 매니저
MQTTとAMQPと.NET
[GKE & Spanner 勉強会] Cloud Spanner の技術概要
오픈소스를 활용한 Batch_처리_플랫폼_공유
SBS 콘텐츠허브의 AWS 도입 사례 :: SBS 콘텐츠허브 :: AWS Media Day 2016
ドメイン駆動設計サンプルコードの徹底解説
20200212 AWS Black Belt Online Seminar AWS Systems Manager
Mongo sharding
Amazon DocumentDB vs MongoDB 의 내부 아키텍쳐 와 장단점 비교
Introducing Change Data Capture with Debezium
검색엔진이 데이터를 다루는 법 김종민
Cluster API によるKubernetes環境のライフサイクル管理とマルチクラウド環境での適用
Kafka as your Data Lake - is it Feasible?
Storing 16 Bytes at Scale
JupyterHub: Learning at Scale
[236] 카카오의데이터파이프라인 윤도영
C* Summit 2013: Eventual Consistency != Hopeful Consistency by Christos Kalan...
Ad

Similar to Ship code like a keptn (20)

PPTX
Making software development processes to work for you
PPTX
Agile & DevOps - It's all about project success
PDF
Cloud and Network Transformation using DevOps methodology : Cisco Live 2015
PPTX
How Azure DevOps can boost your organization's productivity
PPTX
DevOps and Build Automation
PDF
JUG Poznan - 2017.01.31
PDF
Next Level DevOps Implementation with GitOps
PPTX
#ESPC19 How to do #DevOps with #SPFx
PDF
Cloud-Native Fundamentals: Accelerating Development with Continuous Integration
PPTX
Model-Based Testing for ALM Octane: Better tests, built faster
PDF
Lessons Learned from Large Scale Adoption of DevOps for IBM z Systems Software
PDF
Technical debt strategy
PPTX
Dev Ops for systems of record - Talk at Agile Australia 2015
PDF
Cloud continuous integration- A distributed approach using distinct services
PPTX
SCM Transformation Challenges and How to Overcome Them
PPTX
Training Bootcamp - MainframeDevOps.pptx
PPTX
Lyra Infosystems - GitLab Overview Deck 2020
PPTX
JavaOne 2015: Top Performance Patterns Deep Dive
PDF
An Introduction to Microservices
PPTX
Deploying more technology to shift from agility to anti-fragility
Making software development processes to work for you
Agile & DevOps - It's all about project success
Cloud and Network Transformation using DevOps methodology : Cisco Live 2015
How Azure DevOps can boost your organization's productivity
DevOps and Build Automation
JUG Poznan - 2017.01.31
Next Level DevOps Implementation with GitOps
#ESPC19 How to do #DevOps with #SPFx
Cloud-Native Fundamentals: Accelerating Development with Continuous Integration
Model-Based Testing for ALM Octane: Better tests, built faster
Lessons Learned from Large Scale Adoption of DevOps for IBM z Systems Software
Technical debt strategy
Dev Ops for systems of record - Talk at Agile Australia 2015
Cloud continuous integration- A distributed approach using distinct services
SCM Transformation Challenges and How to Overcome Them
Training Bootcamp - MainframeDevOps.pptx
Lyra Infosystems - GitLab Overview Deck 2020
JavaOne 2015: Top Performance Patterns Deep Dive
An Introduction to Microservices
Deploying more technology to shift from agility to anti-fragility
Ad

Recently uploaded (20)

DOCX
The AUB Centre for AI in Media Proposal.docx
PPTX
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
PDF
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
PDF
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
PDF
CIFDAQ's Market Insight: SEC Turns Pro Crypto
PDF
GamePlan Trading System Review: Professional Trader's Honest Take
PDF
Approach and Philosophy of On baking technology
PDF
Reach Out and Touch Someone: Haptics and Empathic Computing
PDF
Advanced Soft Computing BINUS July 2025.pdf
PDF
Unlocking AI with Model Context Protocol (MCP)
PDF
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
PPTX
20250228 LYD VKU AI Blended-Learning.pptx
PDF
solutions_manual_-_materials___processing_in_manufacturing__demargo_.pdf
PPTX
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
PDF
The Rise and Fall of 3GPP – Time for a Sabbatical?
PDF
Empathic Computing: Creating Shared Understanding
PDF
Chapter 3 Spatial Domain Image Processing.pdf
PDF
Network Security Unit 5.pdf for BCA BBA.
PDF
Advanced IT Governance
PPTX
PA Analog/Digital System: The Backbone of Modern Surveillance and Communication
The AUB Centre for AI in Media Proposal.docx
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
CIFDAQ's Market Insight: SEC Turns Pro Crypto
GamePlan Trading System Review: Professional Trader's Honest Take
Approach and Philosophy of On baking technology
Reach Out and Touch Someone: Haptics and Empathic Computing
Advanced Soft Computing BINUS July 2025.pdf
Unlocking AI with Model Context Protocol (MCP)
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
20250228 LYD VKU AI Blended-Learning.pptx
solutions_manual_-_materials___processing_in_manufacturing__demargo_.pdf
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
The Rise and Fall of 3GPP – Time for a Sabbatical?
Empathic Computing: Creating Shared Understanding
Chapter 3 Spatial Domain Image Processing.pdf
Network Security Unit 5.pdf for BCA BBA.
Advanced IT Governance
PA Analog/Digital System: The Backbone of Modern Surveillance and Communication

Ship code like a keptn

  • 1. Shipping code like a @keptnProject https://keptn.sh https://github.com/keptn Rob.Jahn@Dynatrace.com Technical Partner Manager & Keptn Advocate
  • 2. 2 Every CIOs vision of their Cloud Native investment
  • 3. 3 The unfortunate reality for most…. 1. Complexity 2. Manual operations 3. Lack information 4. People 5. Identifying root cause
  • 4. 4* based on our ACM survey
  • 5. 5 Maturity Questions Mean Time to Innovation (MTTI) 1. Are you on two week sprints?​ 2. Do you also release 25 times a year?​ 3. How long does it take you to deploy a new release to production?​ 4. Do you measure feature adoption? Mean Time to Remediate (MTTR) 1. How long does it take to rollback/roll forward a faulty release?​ 2. To release any software in your organization, how many manual steps does it take?​
  • 6. 6 MTTI Mean Time to Innovation MTTR Mean Time to Remediate 4.8 days 4 hours ~ 10min 12.5 days 2 days ~ 1 hour The reality and evidence supports it! https://dynatrace.ai/acsurvey Only < 5% is „Cloud Native“
  • 7. 7 That is why we are building Because cloud native delivery and operations is a BIG challenge for enterprises! Cloud Native
  • 9. 9 Common Pipeline delivery challenges 1. Pipelines are semi-automated 2. Each team building duplicate delivery & integration mechanisms 3. Changing tooling causes a ton pipeline re-work 4. Lack software architecture for DevOps automation 5. Lack auditing support for customized tool integrations 6. Lack of standards 7. Too many technologies to keep up with 8. What else ????
  • 10. 10 So what is this thing? • Open source initiative to make continuous delivery and operations automation a commodity for modern cloud-native applications • Enterprise-grade event-based framework for shipping and operating cloud- native applications built to run on any Kubernetes cluster • With Keptn ... • Developers can focus on code instead of creating YAML files • DevOps can focus on tools instead of building pipelines • SREs can enforce processes instead of debugging problems
  • 11. 11 Automated OperationsContinuous Deployment Core Capabilities -- accelerates building autonomous clouds Event-drivenrunbook automation Productionproblemscan beautomatically remediatedin real-timebyexecuting runbooksthatrequirenomanual intervention. Self-healing blue/green deployments Deploymentsthatfollowthe“Operationsas Code”paradigmautomaticallyremediate problemsandgetyourdeploymentpipeline workingagainin underaminute. Automatedmulti-stage unbreakable pipelines GitOps-enableddeliverypipelineswith automatedqualitygatessupportautomated testingandmonitoring-as-a-service.
  • 12. 12 First: solves the Continuous Delivery Problem! Many approaches today are brittle, error-prone and hard to maintain
  • 13. 13 Quote: “We spend more time in manual communicaton than remediating issues“ Second: has Continuous Operations at its Core! ENGAGE TRIAGE FIND & ASSEMBLE RESOLVE RESTORE MANUAL COMMUNICATION MANUAL COMM Before After RESTORERESOLVE NUMBER OF ISSUES BEFORE: mostly manual AFTER: mostly automated
  • 14. 14 Increase Quality & Level of Automation Increase Speed & Reduce Costs Automated Testing Continuous Performance Auto Quality Gates AUTOMATE OPERATIONS AUTOMATE DEPLOYMENT AUTOMATE MONITORING Feature Flagging Adaptive Scaling Auto Roll-Back Canary Releases Blue / Green Deploymen ts Auto- Remediation AUTOMATE QUALITY Strategically Used as Pipeline Feature Building Blocks for ACM/Cloud Natives!
  • 15. 15 Design Principals – follows patterns for modern applications GitOps-based collaboration Operator patterns for all logic components Monitoring and operations ascode Built onandfor Kubernetes Event-drivenandserverless Pluggabletooling
  • 16. 16 Keptn architecture Platform keptn cli api eventbroker REST API bridge datastore gitconfiguration-service internal- keptn- services distributor distributor keptn services Continuous Delivery Container Registry Operations Automation AIOps GitOps Test Automation ChatOps Data Provider dev staging production keptnNamespaces: distributor Mini-kube AWS EKS Azure AKS Google GKE Openshift Pivotal PKS
  • 17. 17 Keptn Use Cases - Installation 1. One-Line Installation: on most popular k8s platforms 2. Zero-Touch Toolchain Integration: No custom tool integrations needed 3. Zero-Touch Cloud Native Services: Enables GitOps event-driven CD/CO for your services
  • 18. 18 Config ChatOps IT Autom Deploy Test Observe One-Line Installation: $ keptn install --platform=[aks, gke, openshift] 1. Keptn installer pod 2. Keptn pods and services(such as logging, keptn bridge,event broker) 3. Keptn routes (such asauthenticator, bridge, gate-keeper,helm service) 4. Keptn channels (such as configuration-changed, deployment-finished, evaluation-done) 5. Istio and knative pods
  • 19. 19 Config ChatOps IT Autom Deploy Test Observe Zero-Touch Toolchain Integration: $ keptn wear uniform uniform.yaml uniform.yaml services: - name: "dockerhub" - name: "jenkins-service" subscribedchannels: - new-artifact - deployment-finished -name: "slack-service”
  • 20. 20 Zero-Touch Toolchain Integration: Config ChatOps IT Autom Deploy Test Observe Pluggable tooling All tools leveraged by keptn can be replaced based on your tool preferences.
  • 21. Keptn Orders Demo -- tool chain ”uniform” Config ChatOps IT Autom Deploy Test Observe • Configuration:GitHub • Deploy:Helm • Chat:Slack& MS Teams • IT Automation:None • Test:Jmeter • Observe:PitometerQualityGates
  • 22. Keptn Orders Demo Setup DockerRegistrywithpre-builtimages GitHub Org Staging Production Update GitHub repositories - front-end - catalog-service - customer-service - catalog-service Dev https://github.com/keptn-orders
  • 23. 23 Keptn Use Cases – Continuous Delivery 1. Re-Think Pipelines: Gone are the days of custom pipeline coding! 2. Automated Multi-Stage Delivery: Risk-Free auto deployment through multi-stage delivery pipelines 3. Automated Quality Gates: stops bad changes before production using Pitometer 4. Self-Healing Blue/Green Deployments: reverts bad changes before impacting end- users 5. Zero-Touch Toolchain Updates: Add/Remove/Replace tools without custom coding 6. Mastering Continuous Delivery: Risk-Free Automated Deployments
  • 24. 24 Config ChatOps IT Autom Deploy Test Observe Re-Think Pipelines: $ keptn create project keptn-sample shipyard.yaml S T A G I N G P R O D DirectUpdateC D Blue/GreenUpdateC D
  • 25. 25 Config ChatOps IT Autom Deploy Test Observe Zero-Touch Cloud Native Services: $ keptn onboard service myservice [values.yaml] S T A G I N G P R O D DirectUpdateC D Blue/GreenUpdateC D PLACEHOLDER PLACEHOLDER
  • 26. As a developer, I want to safely deploy a new version by just providing my docker image
  • 27. 27 Config ChatOps IT Autom Deploy Test Observe Automated Multi-Stage Delivery: $ keptn new artifact myservice:1.0.0 S T A G I N G P R O D ScoreDirect PerformanceUpdate Promote?C D T O ScoreBlue/GreenUpdate Keep?C D T O PLACEHOLDER PLACEHOLDER 1.0.0 1 1 90 / 100 1.0.0 1 1 1 75 / 100 P R O M O T E K E E P
  • 28. 28 Config ChatOps IT Autom Deploy Test Observe Automated Quality Gates: $ keptn new artifact myservice:2.0.0 S T A G I N G P R O D ScoreDirect PerformanceUpdate Promote?C D T O ScoreBlue/GreenUpdate Keep?C D T O 1.0.0 1 1 45 / 100 1.0.0 1 1 1 2.0.0 2 2 A B O R T
  • 29. Pitometer ??? devices used to measure a ship's speed relative to the water. They are used on both surface ships and submarines. Data from the pitometer log is usually fed directly into the ship's navigation system. (Wikipedia)
  • 30. 30 A Quick word on Keptn Quality Gates: Automated Deployment Validation Metric Source & Query Grading Details & Metric Score Pitometer Specfile Total Scoring Objectives 2GB Allocated Bytes (from Prometheus) > 2GB: 0 Points < 2GB: 20 Points 5% 2% < 2%: 0 Points < 5%: 10 Points > 5%: 20 Points Conversion Rate (Dynatrace) GraderSource If value: 3GB Score: 0 If value: 3.9% Score: 10 Total Score: 10
  • 31. 31 Automated Quality Gates with Keptn’s Pitometer #1 Call Pitometer Library Pass / Fail Promote? Quality Gates as Code • First extensions by https://www.dynatrace.com/news/blog/automated-deployment-and-architectural-validation-with-pitometer-and-keptn/
  • 32. 32 Pitometer Client  Evaluates & Scores results  Community can build more  1 to many modules as required  Queries data for requested indicators  Community can build more Pitometer Module  Core library for Data, sources, graders and perfspec file parsing PerfSpec JSON Grader Module Graders Graders Data Module(s) Custom Code  Logic to assemble and configure modules. Read file and pass to Pitometer functions Pitometer JSON Results Keptn Pitometer: Extensible design using NodeJS Modules https://github.com/keptn/pitometer
  • 33. 33 Quality Gate Examples and Metrics Check 1 • Is bad coding leading to higher costs? Check 2 • New dependencies? On Purpose? • Services connecting accurately? • Number of container instances needed? Check 3 • Are we jeopardizing our SLAs? • Does load balancing work? • Difference between Canaries? Check 4 • Did we introduce new “hidden” exceptions? Metrics  Memory usage  Bytes sent / received  Overall CPU  CPU per transaction type Metrics  Number of incoming / outgoing dependencies  Number of instances running on containers  Metrics  Response Time (Percentiles)  Throughput & Perf per Instance / Canary  Metrics  Total Exceptions  Exceptions by Class & Service
  • 34. 34 Config ChatOps IT Autom Deploy Test Observe Self-Healing Blue/Green Deployments: $ keptn new artifact myservice:3.0.0 S T A G I N G P R O D ScoreDirect PerformanceUpdate Promote?C D T O ScoreBlue/GreenUpdate Keep?C D T O 1.0.0 1 1 85 / 100 1.0.0 1 1 1 2.0.0 2 2 3.0.0 3 3 3.0.0 3 3 3 80 / 100 P R O M O T E R E V E R T
  • 35. As a DevOps Engineer, I want switch tools without writing custom pipeline code
  • 36. 36 Config ChatOps IT Autom Deploy Test Observe Zero-Touch Toolchain Updates: $ keptn update uniform <+neo,+end2end,+spinnaker> S T A G I N G P R O D ScoreDirect PerformanceUpdate Promote?C D T O ScoreBlue/GreenUpdate Keep?C D O 1.0.0 1 1 1.0.0 1 1 1 2.0.0 2 2 3.0.0 3 3 End2EndT
  • 37. 37 Config ChatOps IT Autom Deploy Test Observe Mastering Continuous Delivery: $ keptn new artifact myservice:4.0.0 S T A G I N G P R O D ScoreDirect PerformanceUpdate Promote?C D T O ScoreBlue/GreenUpdate Keep?C D O 1.0.0 1 1 1.0.0 1 1 1 2.0.0 2 2 3.0.0 3 3 4 4 4 End2EndT 4.0.0 4.0.0 4 4 95 / 100 90 / 100 K E E P P R O M O T E
  • 38. 38 Keptn Use Cases – Continuous Operations 1. Self-Healing Production: Automated Problem Remediation (Scale-Up, Scale- Down, ...) 2. Self-Healing/Continuous BizDevOps: Automated Business Operations Optimization, Turn on/off feature flags based on conversion rates ... 3. Auto-Protect Production & Business: Stop DDOS attacks, Redirect Bot Traffic ... 4. Chaos-Driven Operation Readiness: Chaos Engineering to validate your production self-healing
  • 39. As an SRE, I want production to auto- remediate problems detected by monitioring
  • 40. 40 4 4 Self-Healing Production: $ keptn new problem <Services, Root Cause> P R O D Evaluate Decide Act Notify 4 Escalate 4 4 #1 Evaluate Problem #3 Execute Action #4 Notify on Success #5 Escalate on Failure#2 Decide on Remediation
  • 41. 41 • https://github.com/keptn/community • Twitter - @keptnproject • Slack - keptn.slack.com • Biweekly community meetings • Zoom https://dynatrace.zoom.us/j/578828410 • Notes https://tinyurl.com/keptn-meeting-notes • Blogs– http://medium.com/keptn • Web – http://keptn.sh • Email- keptn@dynatrace.com Jointhe community Rob.Jahn@Dynatrace.com Technical PartnerManager & KeptnAdvocate
  • 42. 42
  • 43. 43 Keptn Orders Application • Order processing application • Web UI with 3 java spring boot microservices with embedded databases • Components pre-compiled, built as a Docker image, and staged on dockerhub.com • All source code can be found at https://github.com/keptn-orders
  • 44. 44 Keptn-Orders GitHub Repo Structure App Repos front-end customer- service order-service catalog- service /jmeter/ basiccheck.jmx front-end_load.jmx /perfspec/ perfspec.json Other files * Source Code * CI Build pipeline files * Dockerfile * Build scripts Same structure as front-end Same structure as front-end Same structure as front-end orders- project Dev Branch /Helm-Charts/ chart values files /templates/ istio virtual gateway YAML service YAML deployment YAML environment gateway YAML Staging Branch Same structure and files as Dev Branch Production Branch Same structure and files as Dev Branch Folders &Files Keptn Repo Folders &Files https://github.com/jahn-keptn-orders-aks2
  • 45. 45 VM Worker Pool NAME SPACES Kubernetes Demo Cluster Setup with Keptn & Deployed Keptn-Orders Application Node 1 Dynatrace Keptn Staging Production front-end front-end customer- service order-service catalog- service bridge OneAgent Operator OneAgent (node 1) front-end customer- service order-service catalog- service gatekeeper Jmeter- service helm-service … Dev front-end customer- service order-service catalog- service Node X

Editor's Notes

  • #7: We did a survey with most of our attendees at PERFORM 2019 in Las Vegas – asking them questions around their cloud native projects. Please take the URL and do the survey as well! I looked at two metrics that tell me how mature an organization is: MTTI: how long does it take to get a newly built and already tested feature from pre-prod into production? 1 hour was the benchmark that our CTO gave us internally! MTTR: how long does it take to remediate any problems that may come up? This should be in the range of minutes, not hours or days! The shocking truth of this survey was that only 5% of our customers are reaping the benefits of new technology. The majority is in the bottom left half where they would have assumed that new technology gets them to the top right „magic quadrant“
  • #9: Lack of change caused by a combination of conflicting motivations, processes, and tooling
  • #13: Many teams we spoken with build their own Deployment Pipelines: Combination of OpenSource & Commercial Tools for Deployment Automation! Testing Pipelines: Combination of OpenSource & Commerical Tools for Test Execution Quality Gates: Most often done manual. Some are investing in automated validation! Auto Remediation: Mostly done manual with a trend towards simply remediation actions
  • #21: Currently, keptn install command is doing this using a default set of keptn services. Coming soon, a uniform file will be declarative and run using the keptn cli
  • #32: We took the implementation from Jenkins and „externalized“ it into a core library of keptn called Pitometer. Pitometer follows the same „Quality Gate as Code“ approach but alows you to pull data from multiple data sources. Each evaluated metric will be scored and the total score will be evaluated with the key objective! Check out the blog and keptn for more details. Pitometer allows you to enable autoamted quality gates in EVERY CI/CD tool