Kanban as code: the Continuous Delivery at LesFurets.com by Dimitri Baeli

Kanban as Code
Code level impact of a Lean mindset

@YourTwitterHandle@YourTwitterHandle@beastiefurets@dbaeli
Insurance aggregator
5 Products
2,5M quotes per year
Paid by contracts initiated
Our product

The engine
• Complex forms
• 40-200 Fields (Expedia is around 10 ﬁelds)
• Risk pricing engine
• 50+ external webservices called in real time
• Collect answers within seconds
• 2B+ ﬁelds mapping per year
• Embedded online subscription module
LesFurets
WebSite
Panel Partners
Data Partners

Competitive Advantage(s)
Product Catalogue :
Number of Panel members
Speed of getting new Panel members
Acquisition price compared to google, …
Quality of the offers presented to the users
Quality of the clients presented to the partners
UX of the WebSite :
Speed of getting a proposal, clarity of the proposal
Avoid mistakes along the form ﬁlling

The IT Team
• 23 Developers & 2 Ops
• 2 Functional Teams & 1 Infrastructure & Team
• 1 Single code base of 500.000 lines
• 40.000 unit tests (low level automated tests)
• 200 selenium tests (browser based automated tests)
•1-3 release per day of 1-10 changes
• 1 branch per feature : released when ready
• Soon known as “Kanban as code”
• 20-30 branches in parallel, 1-10 released per day
@beatiefurets
github.com/lesfurets

500.000 Lines of Code (LoC) that’s :
7 dictionaries of 500 pages (140 lines / page)
2 times the 7 Harry Poter volumes (3400 pages)
A new version daily
5-10 changes by 25 devs
Intermezzo : 500.000 line of code ?
x 7
x 2

Why daily releases ?
What read and understood

Manifeste agile
Principe #1
« Our highest priority is to satisfy the customer through early
and continuous delivery of valuable software. »
http://agilemanifesto.org/principles.html

Food for thought : back to the future

Reading survivor kit : save your life

Minimal Viable : what you can’t ignore

Release early, release often
http://paulhammant.com/2013/03/13/facebook-tbd-take-2/

Learning #1 : Release early, release often
• To reduce the Time to Market :
• Release small batches of features
• Release often
• Releasing often
• Change the way source code is managed
• Change in the QA strategy (Dev QA their own changes)
• Test tooling is provisioned for the des
• Fast Feedbacks as a gift

Continuous Delivery : 5 Golden Rules
1. Fast build
2. Unbreakable build
3. Easy and automated deployments
4. Runtime Monitoring and alerting
5. Root cause analysis of incidents

Git / Git Flow / Github Flow
Powerful branching model
http://nvie.com/posts/a-successful-git-branching-model/
Master
Branch
Pull
Request
Github

Code management in 2017
The minimum to know, no excuse !

@beastiefurets@dbaeli
Original code:
master

Change 1 : Lean IT

Change 2 : Beastie Furets

Change 3 : Hello Beastie

A commit

master
Commit 1 : Lean IT
commit 1

A branch (copy)

Change 2 = commit 2
commit 2
master commit 1

Let’s merge the branch

master
Change 1 & Change 2 : combined (merged)
commit 1
merged
commit 2

A conflict of 2 branches

Commit 3 : Hello Beastie
master
commit 3

master
Commit 2 : Hello Lean IT
commit 2

???????
Change 3 : Hello Beastie
master
conflict
commit 3
commit 2

Minimum viable vocabulary (1/2)
•Commit:
• is a change within the code base
• described as a “diff” from original (or patch)
• History :
• Sequence of Commits (changes)
• Branch:
• A full copy of the code base at a given time
• Can be seen as an alternate history
• Merge:
• Replay commits of a branch on another
branch 1
branch
commit 1
commit 2
commit 4
commit
merge
commit 3

Minimum viable vocabulary (2/2)
• Conﬂict:
• when 2 commits aren’t compatible
• from 2 branches
• Resolution :
• give a solution to the conﬂict
• Rebase :
• change the start point of a branch
• time travel (get into the future)
branch 1
branch 2 commit 2
commit a
conflict
commit 3
commit 2
commit a
commit 3
rebase

Release more often - Step 1
Monthly releases (2012)

Plan / estimate / code / test / release monthly
Sprints style
12 releases in a year
2012 : Monthly release

branch (prod)
trunk trunk
3 - 4 semaines 1 - 4 jours5 jours
Release
Code Freeze
branch

Storie 1
branch (prod)
trunk trunk
Release
Code Freeze
branch

Storie 1
branch (prod)
trunk trunk
Storie 2
Release
Code Freeze
branch

Storie 1
branch (prod)
trunk trunk
Storie 2
Release
Code Freeze
branch
Risk of conflicts

Agile Team of 12, SCRUM by the book
•Monthly sprints, planning, coding, testing, demo, release
• 1 Week of testing (code review, manual testing, bug ﬁxes)
• Build Time : 15 minutes
• Regression Testing : 1-2 hour
•Blockers :
• Build time
• Regression Testing
• A Week of manual testing
2012 : Monthly releases

Weekly release (2013)

2013 : Weekly release
Planiﬁer / estimer / coder / tester
Release weekly
50 releases per year

branch (prod)
trunk trunk
Release
Code Freeze
branch
Storie 1

Storie 1
branch (prod)
trunk trunk
Storie 2
Release
Code Freeze
branch
3 - 4 weeks 1 - 4 days5 days
1 week

• Keep Scrum monthly planning with Weekly releases
•Changes:
• Use the patch process to deliver features (cherry pick)
• Build down to 3 minutes (vs 15 minutes)
•Blockers:
• Regression Testing + Recette
• Code level management (release branch)
• Painful weekly patch release
2013 : Getting Weekly

Learning #2 : Patch process as release process
•When something goes wrongs on the live system
•Do you have a strong process to release a ﬁx ?
• NO => you’re living dangerously => get one
• YES => why don’t you use it for usual releases ?
•Anyway :
• Get a robust and fast delivery process is not an option !

Daily release (2014)

2014: Daily Releases
Ship what is ready … every day !
Marathon style
250 releases per year (already 50 in 2017 !)

master
trunk
1 day 1 day
Model 3: Kanban as code
Storie 1
Storie 2
Storie 3

master
trunk
1 day 1 day
Storie 1

master
trunk
1 day 1 day
Storie 1
Is this branch ready ?

master
trunk
1 day 1 day
Storie 1
merge & test
Merge the branch and test

master
trunk
1 day 1 day
Storie 1
merge & test & drop
Merge the branches and test

master
trunk
1 day 1 day
Storie 1
Storie 2
merge & test

master
trunk
1 day 1 day
Storie 1
Storie 2
Storie 3
merge & test

Keep the branches independent
Avoid the conflict
git-octopus by lesfurets (on GitHub)

master
trunk
1 day 1 day
Storie 1
Storie 2
Storie 3

ticket1
ticket2
ticket3
ticket4
ticket5
features releaseslocal
ticket3
master
ticket3
ticket1
master
octopus-features
octopus-releases
few minutes 1 jour à 1 mois 1 - 2 jours
Validation
0 - 2 jours
release

Learning #4 : Don’t block the others
•We keep the features separated until the last moment
• Other teams are merging by default !!!
• Continuous Merge (as test tooling):
• Detect conflicts with others (without requiring you to get their code)
• Allow you to avoid conflits
• Allow you to drop your code temporarely
•Bonus:
• Avoid unfinished code to be used by the others

Break release branches, release from trunk
• Improvments:
• Automated release from ﬁnished development
• Per feature code review & demo
• 15 minutes regression testing (testing architecture)
• Release by 1 dev alone
• Monitoring & Root cause analysis of issues
2014: Daily Releases

Learnings : Kanban as Code
#1 : Release early, release often
#2 : Patch process as main release process
#3 : Avoid merge & conflicts
#4 : Don’t block the others

Take away : Kanban as Code
The Goals:
• # A Flow of Features at code level (Branches)
• # Release what is ready when it’s ready (Start by ﬁnishing)
• # Branches are your Kanban card (Step 1 of a pull system)
Why does it work :
• # Enforce the features to be independent (Really Independent !)
• # Low risk of conﬂict on 500.000 Line of Code (daily)
• # No branch can block another one (or it the same)

Beastie Furets (LesFurets IT Team)
@beastiefurets
beastie.lesfurets.com
lesfurets

@dbaeli
@beastiefurets
MERCI !

69
LesFurets.com LeanKanban.fr29, 30 Novembre 2017
www.leankanban.fr

Avoid the conflict

Change 3 : Original (conflict with Change 2)
master
commit 3

Change 3bis : New version (avoid conflict)
master
commit 3
commit 2

Commit 1, 2: Easy merge
master
commit 3
commit 2
resolved
commit 1

Change 1, 2 & 3bis : Merged
master
merged
commit 3
commit 2
commit 1

Learning #3: Avoid merge & conflicts
• For 2 Features developed separately:
• Merge = enforce to release together
• But Merging new code is the default for many teams
•Avoid merges (or release immediately):
• Trunk Based Development
• Any alternatives ?

Kanban as code: the Continuous Delivery at LesFurets.com by Dimitri Baeli

More Related Content

Viewers also liked (19)

Similar to Kanban as code: the Continuous Delivery at LesFurets.com by Dimitri Baeli (20)

More from Institut Lean France (20)

Recently uploaded (20)

Kanban as code: the Continuous Delivery at LesFurets.com by Dimitri Baeli