SlideShare a Scribd company logo
Transforming Chaos to Clarity
Making Your Software Development
Hum
Ron Lichty, Software Engineering Mgmt
Ron@RonLichty.com
Ron Lichty,
Managing Software People & Teams
SOFTWEST
© Ron Lichty 2
* http://InformIT.com/managing <-----35% off coupon code: MANTLELICHTY
http://ManagingTheUnmanageable.net <-----tools, excerpts, more rules of thumb
*
Poll: Software Development
in Disarray?
• Who has seen chaos in a product group?
• Who has seen chaos in your current group?
• Who feels like development is running
rough?
• Who is suffering from organizational knots?
• Anyone’s development organization running
so smoothly it doesn’t need tuning?
Define Success
• What are you trying to accomplish
• How do you know you're not
• How will you know when you get there
• Assess what’s working
• Assess the issues and the symptoms
– Every organization is unique
Issues and Symptoms
• Turnover
• Productivity
• Handoffs
• Process glitches, Bottlenecks
• Quality
• Single points of failure
• Communication breakdowns
• Unfeasible sales
• Heroes vs Teamwork
• Sources of disruption and interruption
No One Can Fix Everything
No One Can Fix Everything
• Life is simpler when you plow
around the stump.
Chaos Isn’t All Bad
Chaos Isn’t All Bad
• Innovation often emerges from “chaos”
– Brainstorming
– Thinking outside the box
• Don’t eliminate useful chaos
• Just look for the pings and the misfires
– Let’s make our product engines hum
• regardless of where creativity and pivots take us
Patterns of Impediments
Categories of Impediments
• Process
• Culture
• Communication
• Planning
• Rigor
Requirements: Bridge the Gap
• GIGO
• Programmers:
– who has received an exceptional set of rqmts?
– what was the programming experience like?
– how did it differ from the usual?
• How good are your requirements?
– How unambiguous?
• Do your requirements change?
• Are they ever complete?
Requirements: Solutions
• Communication
• Team-building
• Agile
15
Blank space.
The Study of Product Team Performance, 2014
Requirements:
Fix Interdepartmental Communication
16
Blank space.
The Good News
10%
of product teams function as a
seamless team.
The Bad News
90%
of product teams don’t.
How would you rate your product team’s cross-functional
collaboration, trust and communication?
Requirements:
Fix Interdepartmental Communication
• Build trust relationships
• Product Mgmt & Engineering: collaboration
• Establish processes for your partners to fit
• Communicate, communicate, communicate
• Never succumb to “them vs. us”
• Avoid the “blame game”
Requirements: Solutions
• Agile
– Teamwork: developers, testers, product managers
• Co-located, with a focus on team interaction
– A single Product Owner
– Just enough requirements
• The entire project at a high level
• Prioritized by business value
• Details “just in time”
– Requirements very stable during sprints
• Change occurs in the sprint boundaries
Agile: The Challenges
• Doing Agile
• Being Agile
Doing Agile: Practices
• Common Definition of Done
• Timeboxing Standups (and every meeting)
• Standups: three questions
• Retrospectives
• Rapid Relative Sizing
• Estimating = ƒ(Size, Velocity, Stable teams)
• Sizing, Estimating Committing≠
• Shippable software every iteration
Being Agile
• It’s the values and the principles
• Half done is not done
• Standups: How are we doing against our plan?
• Being empowered and self-organizing
• Being a learning organization
• Delivering highest customer value every
sprint
• Building trust and respect with each other
Focus
• Macro level
• “When everything is a priority,
nothing is a priority.” --Sheila Brady, Apple
• Limit task switching
• Micro level
• Avoid “multitasking”
• Limit interruptions
Task Switching
Source: Mike Cohn, citing Steven C.Wheelwright and Kim B.Clark (1993), Revolutionizing Product Development: Quantum Leaps in Speed, Efficiency and Quality, 1993
Task Switching
Source: Rob Maher, “Increasing Team Productivity: A project focus creates waste and leaves value on the table”, Scrum.org Whitepapers
taking communications overload into account
Rob Maher, surmising that email, texts, IRC, chat, smartphones together represent a second “task”
Source: Mike Cohn, citing Steven C.Wheelwright and Kim B.Clark (1993), Revolutionizing Product Development: Quantum Leaps in Speed, Efficiency and Quality, 1993
--Larry Maccherone, The Impact of Agile Quantified, Rally, 2013
Focus
• Macro level
• “When everything is a priority,
nothing is a priority.” --Sheila Brady, Apple
• Limit task switching
• Micro level
• Avoid “multitasking”
• Limit interruptions
It’s not that we don't have enough time.
We have too much to do.
-- Chet Hendrickson
Smart Project Planning
• Show demonstrable progress frequently
• Get the risky stuff done first
– the UI should always be high on the risk list
• Deliver the highest customer value first
• Be first / be ready to integrate / be early
• Don’t over-engineer
Pay Down Technical Debt!
• What’s “technical debt”?
• Result: interest accrues
• Solutions?
– Pay down debt
– Prioritize
– Refactor
– Write tests
– Embrace Test Driven Development (TDD)
Optimize Process
• Just about any process is better than no
process
– Mark Ginnebaugh
– The exception: process for process’ sake
• I’m a fan of
– “Just-Enough Process”
– Agile
– baby steps
Improve Onboarding
Improve Onboarding
• Effective Team Onboarding:
– 1 of 5 factors
• 67% likelihood high level team performance
32
Only 4% of organizations indicate they have a best practice
onboarding product team members
Improve Onboarding
The Bottom Line
• Chaos is common
• It’s really a series of challenges
• It’s a series of improvement milestones
• Each of them can be transformed
– Likely each new hum will reveal the next ping
• Like peeling an onion or climbing a mountain
In the beginning, everyone will talk
about scope, and budget, and
schedule, but in the end, nobody
really cares about any of those
things.
The only thing they care about is this:
People will love your software,
or they won’t.
So that’s the only criterion to which
you should truly manage.
—Joseph Kleinschmidt
Ron Lichty Consulting
• Mentoring, coaching, interim VP Engineering, acting CTO:
– http://ronlichty.com, Ron@RonLichty.com
• The book:
Managing the Unmanageable:
Rules, Tools & Insights for Managing Software People & Teams
– http://ManagingTheUnmanageable.net <-----tools, excerpts, more rules of thumb
– http://InformIT.com/managing <---35% off coupon code: MANTLELICHTY
• The annual study:
The Study of Product Team Performance
– http://ronlichty.com/study.html <----- the studies are free downloads
• Training:
Zero to Agile in Three Days
The Agile Manager
Managing Software People and Teams
© Ron Lichty 35
Q&A
Ron Lichty
Ron@RonLichty.com
http://www.ronlichty.com

More Related Content

PDF
Teamwork - making your dream team come true
PDF
Critical Agile Practices, Nuanced Techniques
PDF
Product Owners - How to get your development team to love you (ProductTankSV,...
PDF
Crash course- managing software people and teams
PPT
Product owners - how to get your development team to love you (product school...
PPT
Crash Course - managing software people and teams (sfelc, 10.26.16)
PDF
Do you want to be a manager (are you sure)
PDF
Transforming Chaos To Clarity, Ron Lichty
Teamwork - making your dream team come true
Critical Agile Practices, Nuanced Techniques
Product Owners - How to get your development team to love you (ProductTankSV,...
Crash course- managing software people and teams
Product owners - how to get your development team to love you (product school...
Crash Course - managing software people and teams (sfelc, 10.26.16)
Do you want to be a manager (are you sure)
Transforming Chaos To Clarity, Ron Lichty

What's hot (20)

PDF
Engineering challenges, product management solutions - product camp 2016
PDF
What's It Take to Be a (Good!) Manager
PPT
Scaling Agile Teams
PDF
AIPMM talk - chaos to clarity: managing the unmanageable, ron lichty, 12.7.12
PDF
12 Take Aways - Managing the Unmanageable
PPT
Ict educators win-win-win w agile, ron lichty, 1.4.13
PPT
The hidden ingredient in hyper productive teams (scrummasters guild, 10.15)
PPT
Leading and Motivating Engineers - what product managers need to know - prod...
PPT
Crash Course: Managing Software People and Teams (IEEE, 4.4.13)
PDF
Product owners - how to get your development team to love you (product tank, ...
PDF
Definitions of Done and High Performance Teams
PDF
Becoming an Agile Manager (bay scrum, 10.24.13)
PPT
12 Take Aways - Managing the Unmanageable
PDF
Dream teams - making your dream (team) come true
PDF
If we are agile, why do we need managers (code camp, 10.14)
PDF
If We Are Agile, Why Do We Need Managers? (AgileIndy, 5.14)
PDF
12 take aways - managing the unmanageable
PPT
Keys to crafting an effective agile culture (svcc, 10.15)
PDF
Agile Is Hard (AgileCampSV 2014)
PDF
Managing Using Intuition and Rules of Thumb 050113
Engineering challenges, product management solutions - product camp 2016
What's It Take to Be a (Good!) Manager
Scaling Agile Teams
AIPMM talk - chaos to clarity: managing the unmanageable, ron lichty, 12.7.12
12 Take Aways - Managing the Unmanageable
Ict educators win-win-win w agile, ron lichty, 1.4.13
The hidden ingredient in hyper productive teams (scrummasters guild, 10.15)
Leading and Motivating Engineers - what product managers need to know - prod...
Crash Course: Managing Software People and Teams (IEEE, 4.4.13)
Product owners - how to get your development team to love you (product tank, ...
Definitions of Done and High Performance Teams
Becoming an Agile Manager (bay scrum, 10.24.13)
12 Take Aways - Managing the Unmanageable
Dream teams - making your dream (team) come true
If we are agile, why do we need managers (code camp, 10.14)
If We Are Agile, Why Do We Need Managers? (AgileIndy, 5.14)
12 take aways - managing the unmanageable
Keys to crafting an effective agile culture (svcc, 10.15)
Agile Is Hard (AgileCampSV 2014)
Managing Using Intuition and Rules of Thumb 050113
Ad

Viewers also liked (11)

PDF
Prioritizing The Product Backlog AIPMM Webinar
PDF
Becoming an Agile Manager (Agile Camp, 9.21.13), by Ron Lichty
PDF
Structured military messaging & NEO Net Enabled Operations
PPTX
Think Like an Agilist - Agile Sydney 2014
PPTX
Prioritise: Kanban Recipe for Success Step 5
PPT
Lean Software Development: On Radiators and Refrigerators
PDF
Lean more than startups, software development, manufacturing
PDF
Agile Australia 2014: Pop-ups and Startups in a Large Enterprise
PPTX
Think Like an Agilist (repeat) Sydney Agile and Scrum 2014
PPTX
Balance Demand Against Throughput: Kanban Recipe for Success Step 4
PPTX
Edgy Lean, Agile, and Systems Thinking things that you may not have heard of
Prioritizing The Product Backlog AIPMM Webinar
Becoming an Agile Manager (Agile Camp, 9.21.13), by Ron Lichty
Structured military messaging & NEO Net Enabled Operations
Think Like an Agilist - Agile Sydney 2014
Prioritise: Kanban Recipe for Success Step 5
Lean Software Development: On Radiators and Refrigerators
Lean more than startups, software development, manufacturing
Agile Australia 2014: Pop-ups and Startups in a Large Enterprise
Think Like an Agilist (repeat) Sydney Agile and Scrum 2014
Balance Demand Against Throughput: Kanban Recipe for Success Step 4
Edgy Lean, Agile, and Systems Thinking things that you may not have heard of
Ad

Similar to Transforming chaos to clarity - acm 6.15 (20)

PDF
Stldodn 2014 agile on a shoestring
PPT
Cross Functional Teams: Looking Beyond Development
PDF
Shape Up Your Agility
PPTX
Organizational Design for Effective Software Development
PDF
6.2 Cross-Functional Team Framework - v2.0
PDF
PHP World DC 2015 - What Can Go Wrong with Agile Development and How to Fix It
PDF
When Management Asks You: “Do You Accept Agile as Your Lord and Savior?” - Ci...
KEY
Thezenofscrum1 090221154550-phpapp01
PPTX
Lean Product Development at Discovery Communications: Methodology, Practices,...
PPTX
PDF
Designing for Agile Delight! Customer Obsessed Innovation at Intuit
PPT
Agile Executive Briefing - Situational Assessment + 50k Ft View
PDF
Fundamentals of Agile
PDF
Deeply Embedding UX Practices Into Your Organization by Grafting them Into Yo...
PDF
Jerry Walker - without a plan
PDF
Practical Scrum - one day training
PDF
Agile 2 Cliff Berg Kurt Cagle Lisa Cooney Philippa Fewell Adrian Lander Raj N...
PDF
Agile 2 Cliff Berg Kurt Cagle Lisa Cooney Philippa Fewell Adrian Lander Raj N...
PDF
Agile Fundamentals for Project Managers.pdf
PPTX
This one weird trick will fix all your Agile problems
Stldodn 2014 agile on a shoestring
Cross Functional Teams: Looking Beyond Development
Shape Up Your Agility
Organizational Design for Effective Software Development
6.2 Cross-Functional Team Framework - v2.0
PHP World DC 2015 - What Can Go Wrong with Agile Development and How to Fix It
When Management Asks You: “Do You Accept Agile as Your Lord and Savior?” - Ci...
Thezenofscrum1 090221154550-phpapp01
Lean Product Development at Discovery Communications: Methodology, Practices,...
Designing for Agile Delight! Customer Obsessed Innovation at Intuit
Agile Executive Briefing - Situational Assessment + 50k Ft View
Fundamentals of Agile
Deeply Embedding UX Practices Into Your Organization by Grafting them Into Yo...
Jerry Walker - without a plan
Practical Scrum - one day training
Agile 2 Cliff Berg Kurt Cagle Lisa Cooney Philippa Fewell Adrian Lander Raj N...
Agile 2 Cliff Berg Kurt Cagle Lisa Cooney Philippa Fewell Adrian Lander Raj N...
Agile Fundamentals for Project Managers.pdf
This one weird trick will fix all your Agile problems

More from Ron Lichty (6)

PDF
Crash Course - Managing Software People and Teams
PDF
What the Heck Is a Product Owner?
PDF
Product owners how to get your development team to love you (product camp, 3...
PDF
If We Are Agile, Why Do We Need Managers? (sv-aln, 7.14)
PDF
If we’re agile, why do we need managers (tri valley aln, 3.14)
PDF
How to get your agile development team to love you (product camp, 3.14)
Crash Course - Managing Software People and Teams
What the Heck Is a Product Owner?
Product owners how to get your development team to love you (product camp, 3...
If We Are Agile, Why Do We Need Managers? (sv-aln, 7.14)
If we’re agile, why do we need managers (tri valley aln, 3.14)
How to get your agile development team to love you (product camp, 3.14)

Recently uploaded (20)

PDF
Building Integrated photovoltaic BIPV_UPV.pdf
PDF
Advanced methodologies resolving dimensionality complications for autism neur...
PDF
Unlocking AI with Model Context Protocol (MCP)
PDF
Review of recent advances in non-invasive hemoglobin estimation
PDF
Mobile App Security Testing_ A Comprehensive Guide.pdf
PDF
The Rise and Fall of 3GPP – Time for a Sabbatical?
PDF
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
PDF
Spectral efficient network and resource selection model in 5G networks
PPTX
Understanding_Digital_Forensics_Presentation.pptx
PDF
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
PDF
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
PDF
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
PPTX
MYSQL Presentation for SQL database connectivity
PDF
NewMind AI Monthly Chronicles - July 2025
PDF
Network Security Unit 5.pdf for BCA BBA.
PDF
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
PPTX
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
PPTX
PA Analog/Digital System: The Backbone of Modern Surveillance and Communication
PPTX
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
Building Integrated photovoltaic BIPV_UPV.pdf
Advanced methodologies resolving dimensionality complications for autism neur...
Unlocking AI with Model Context Protocol (MCP)
Review of recent advances in non-invasive hemoglobin estimation
Mobile App Security Testing_ A Comprehensive Guide.pdf
The Rise and Fall of 3GPP – Time for a Sabbatical?
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
Spectral efficient network and resource selection model in 5G networks
Understanding_Digital_Forensics_Presentation.pptx
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
MYSQL Presentation for SQL database connectivity
NewMind AI Monthly Chronicles - July 2025
Network Security Unit 5.pdf for BCA BBA.
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
Dropbox Q2 2025 Financial Results & Investor Presentation
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
PA Analog/Digital System: The Backbone of Modern Surveillance and Communication
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx

Transforming chaos to clarity - acm 6.15

  • 1. Transforming Chaos to Clarity Making Your Software Development Hum Ron Lichty, Software Engineering Mgmt Ron@RonLichty.com
  • 2. Ron Lichty, Managing Software People & Teams SOFTWEST © Ron Lichty 2
  • 3. * http://InformIT.com/managing <-----35% off coupon code: MANTLELICHTY http://ManagingTheUnmanageable.net <-----tools, excerpts, more rules of thumb *
  • 4. Poll: Software Development in Disarray? • Who has seen chaos in a product group? • Who has seen chaos in your current group? • Who feels like development is running rough? • Who is suffering from organizational knots? • Anyone’s development organization running so smoothly it doesn’t need tuning?
  • 5. Define Success • What are you trying to accomplish • How do you know you're not • How will you know when you get there • Assess what’s working • Assess the issues and the symptoms – Every organization is unique
  • 6. Issues and Symptoms • Turnover • Productivity • Handoffs • Process glitches, Bottlenecks • Quality • Single points of failure • Communication breakdowns • Unfeasible sales • Heroes vs Teamwork • Sources of disruption and interruption
  • 7. No One Can Fix Everything
  • 8. No One Can Fix Everything • Life is simpler when you plow around the stump.
  • 10. Chaos Isn’t All Bad • Innovation often emerges from “chaos” – Brainstorming – Thinking outside the box • Don’t eliminate useful chaos • Just look for the pings and the misfires – Let’s make our product engines hum • regardless of where creativity and pivots take us
  • 12. Categories of Impediments • Process • Culture • Communication • Planning • Rigor
  • 13. Requirements: Bridge the Gap • GIGO • Programmers: – who has received an exceptional set of rqmts? – what was the programming experience like? – how did it differ from the usual? • How good are your requirements? – How unambiguous? • Do your requirements change? • Are they ever complete?
  • 15. 15 Blank space. The Study of Product Team Performance, 2014
  • 16. Requirements: Fix Interdepartmental Communication 16 Blank space. The Good News 10% of product teams function as a seamless team. The Bad News 90% of product teams don’t. How would you rate your product team’s cross-functional collaboration, trust and communication?
  • 17. Requirements: Fix Interdepartmental Communication • Build trust relationships • Product Mgmt & Engineering: collaboration • Establish processes for your partners to fit • Communicate, communicate, communicate • Never succumb to “them vs. us” • Avoid the “blame game”
  • 18. Requirements: Solutions • Agile – Teamwork: developers, testers, product managers • Co-located, with a focus on team interaction – A single Product Owner – Just enough requirements • The entire project at a high level • Prioritized by business value • Details “just in time” – Requirements very stable during sprints • Change occurs in the sprint boundaries
  • 19. Agile: The Challenges • Doing Agile • Being Agile
  • 20. Doing Agile: Practices • Common Definition of Done • Timeboxing Standups (and every meeting) • Standups: three questions • Retrospectives • Rapid Relative Sizing • Estimating = ƒ(Size, Velocity, Stable teams) • Sizing, Estimating Committing≠ • Shippable software every iteration
  • 21. Being Agile • It’s the values and the principles • Half done is not done • Standups: How are we doing against our plan? • Being empowered and self-organizing • Being a learning organization • Delivering highest customer value every sprint • Building trust and respect with each other
  • 22. Focus • Macro level • “When everything is a priority, nothing is a priority.” --Sheila Brady, Apple • Limit task switching • Micro level • Avoid “multitasking” • Limit interruptions
  • 23. Task Switching Source: Mike Cohn, citing Steven C.Wheelwright and Kim B.Clark (1993), Revolutionizing Product Development: Quantum Leaps in Speed, Efficiency and Quality, 1993
  • 24. Task Switching Source: Rob Maher, “Increasing Team Productivity: A project focus creates waste and leaves value on the table”, Scrum.org Whitepapers taking communications overload into account Rob Maher, surmising that email, texts, IRC, chat, smartphones together represent a second “task” Source: Mike Cohn, citing Steven C.Wheelwright and Kim B.Clark (1993), Revolutionizing Product Development: Quantum Leaps in Speed, Efficiency and Quality, 1993
  • 25. --Larry Maccherone, The Impact of Agile Quantified, Rally, 2013
  • 26. Focus • Macro level • “When everything is a priority, nothing is a priority.” --Sheila Brady, Apple • Limit task switching • Micro level • Avoid “multitasking” • Limit interruptions It’s not that we don't have enough time. We have too much to do. -- Chet Hendrickson
  • 27. Smart Project Planning • Show demonstrable progress frequently • Get the risky stuff done first – the UI should always be high on the risk list • Deliver the highest customer value first • Be first / be ready to integrate / be early • Don’t over-engineer
  • 28. Pay Down Technical Debt! • What’s “technical debt”? • Result: interest accrues • Solutions? – Pay down debt – Prioritize – Refactor – Write tests – Embrace Test Driven Development (TDD)
  • 29. Optimize Process • Just about any process is better than no process – Mark Ginnebaugh – The exception: process for process’ sake • I’m a fan of – “Just-Enough Process” – Agile – baby steps
  • 31. Improve Onboarding • Effective Team Onboarding: – 1 of 5 factors • 67% likelihood high level team performance
  • 32. 32 Only 4% of organizations indicate they have a best practice onboarding product team members Improve Onboarding
  • 33. The Bottom Line • Chaos is common • It’s really a series of challenges • It’s a series of improvement milestones • Each of them can be transformed – Likely each new hum will reveal the next ping • Like peeling an onion or climbing a mountain
  • 34. In the beginning, everyone will talk about scope, and budget, and schedule, but in the end, nobody really cares about any of those things. The only thing they care about is this: People will love your software, or they won’t. So that’s the only criterion to which you should truly manage. —Joseph Kleinschmidt
  • 35. Ron Lichty Consulting • Mentoring, coaching, interim VP Engineering, acting CTO: – http://ronlichty.com, Ron@RonLichty.com • The book: Managing the Unmanageable: Rules, Tools & Insights for Managing Software People & Teams – http://ManagingTheUnmanageable.net <-----tools, excerpts, more rules of thumb – http://InformIT.com/managing <---35% off coupon code: MANTLELICHTY • The annual study: The Study of Product Team Performance – http://ronlichty.com/study.html <----- the studies are free downloads • Training: Zero to Agile in Three Days The Agile Manager Managing Software People and Teams © Ron Lichty 35

Editor's Notes

  • #4: Managing the Unmanageable: Rules, Tools, and Insights for Managing Software People and Teams http://ManagingTheUnmanageable.net &amp;lt;-----tools, excerpts, more rules of thumb http://InformIT.com/managing   &amp;lt;-----35% off coupon code: MANTLELICHTY
  • #24: Mike Cohn: citing Steven C.Wheelwright and Kim B.Clark (1993), Revolutionizing Product Development: Quantum Leaps in Speed, Efficiency and Quality, 1993
  • #25: Mike Cohn: citing Steven C.Wheelwright and Kim B.Clark (1993), Revolutionizing Product Development: Quantum Leaps in Speed, Efficiency and Quality, 1993