Manchester ITExpo Talk: DevOps large and small - Cambridge Satchel
- 1. jmwprojmwpro
- 2. jmwpro Web Master / Lead Java Developer Lead Developer / Head of Development Director of Platform Development Head of Web Engineering Chief Technology Officer2014 2011 2007 2003 1999 Background DevOps Large DevOps Small DevOps Small ? Small ???? Small
- 3. jmwpro DevOps Large <aka DevOps Hard/>
- 4. jmwpro • 2011-2014 introduced Continuous Integration, DevOps & Behaviour Driven Development to M&S on £150 Million project. • 650 Member project team, mostly outsourced. • Project touched 65 new or modified applications, mostly legacy. • Devops Capability team now in place to kick start other teams • BDD initiatives taken off on and offshore with 4 years to re-platform – ongoing…. DevOps Large: Marks & Spencer Founded 1884, 85,000 staff £10.3 Bn group revenues
- 5. jmwpro “It’s all about the code” Application code, Test code, Configuration code, Script code, infrastructure code, 3rd Party Binaries
- 6. jmwpro DevOps Large: The Good Software Factory Efficiency Ensure there is only one place to put code, and one way to build it Software Scanning Quality and Maintainability Ensure all code meets agreed coding standards. Scan for Security vulnerabilities after every commit Route to Production Simplicity and Control All code had to go through the correct environments first Regular Release train(s) Team Productivity Have a schedule and stick to it. Configuration Management Code control SVN misused, sorted then moved to GIT Behavior Driven Development Better solution. Automated testing. Try BDD initiatives within in-house teams- then roll out offshore. Initiative Benefit What to look out for?
- 7. jmwpro DevOps Large: The Bad Architecture Too coupled Too many dependencies between systems making it difficult to move at speed or test independently Vendors and Systems Integrators No Unit testing, test automation, DevOps Out of the box Vendor code poor with no or limited unit tests. Teams aligned to build features not an efficient pipeline. Automated Testing Multiple frameworks not fit for pipeline Ensure your automated test ‘frameworks’ can run externally (ie via jenkins or similar) and can be configured to run subsets of test. SCM. Legacy Systems Legacy Systems!! Legacy Systems!!! Add a legacy system being built by a legacy Team and the whole code delivery process can grind to a standstill. Test Environments Complex and Expensive Do all you can to build your apps independently of the dreaded “System Integration Environment” Initiative Benefit Explanation
- 8. jmwpro DevOps Large: The Ugly Contracts Fixed Price No mention of operational considerations; code process, build mechanisms, frequency of deployment, code quality, automation. Hosting Partners No Access to hardware Ensure you’re not handing over full control of infrastructure to 3rd Parties with no incentive to give dev/test teams access. What’s the incentive to collaborate. Operating Model Waterfall based Internal processes/gates/sign-off based on waterfall. DevOps initiatives cut across all phases of standard projects. Architecture & Test Data Impossible to get baseline data With multiple batch jobs and internal reference tables virtually impossible to take co-ordinated snapshot of DBs and recreate test environments at will Initiative Benefit Explanation
- 9. jmwpro Lies, Lies and Statics <the forgotten teams/>
- 10. jmwpro The team’s level of agile/lean working practices Level of Independently testable and deployable software Low High Low High Slow Fast Continuous Delivery Daily/Weekly Independent Monthly Coordinated Quarterly Enterprise TEAM TECHNOLOGY The cool kids in DevOps Soon ;-)
- 11. jmwpro The team’s level of agile/lean working practices Level of Independently testable and deployable software Low High Low High Slow Fast Continuous Delivery Daily/Weekly Independent Monthly Coordinated Quarterly Enterprise TEAM TECHNOLOGY Legacy Zone Cool Zone Team Software Agile/Lean practices Great Software Good practices Good Software Poor working practices Poor Software Bad working practices Bad Software Delivering better software, faster.
- 12. jmwpro The team’s level of agile/lean working practices Level of Independently testable and deployable software Low High Low High Slow Fast Continuous Delivery Daily/Weekly Independent Monthly Coordinated Quarterly Enterprise TEAM TECHNOLOGY The real cost of not doing it right $$$$ $$$ $$ $ Understand the cost to the organisation of slow releases Cost of rework Cost of delay and hand off Cost of building the wrong thing Cost of poor technology choice Integration test costs
- 13. jmwpro The team’s level of agile/lean working practices Level of Independently testable and deployable software Low High Low High Slow Fast Continuous Delivery Daily/Weekly Independent Monthly Coordinated Quarterly Enterprise TEAM TECHNOLOGY Legacy Zone Cool Zone Excelling at Software Delivery Delivering better software, faster. Corporate Complexity So where on the scale are you?
- 14. jmwpro The team’s level of agile/lean working practices Level of Independently testable and deployable software Low High Low High Slow Fast Continuous Delivery Daily/Weekly Independent Monthly Coordinated Quarterly Enterprise TEAM TECHNOLOGY Legacy Zone Cool Zone DevOps Big: Enterprise Reality DevOps / BDD Culture on a few new initiatives Outsourced Standard Methods long integration test phases, some test automation Increasing use of BDD, Lean within digital teams
- 15. jmwpro The team’s level of agile/lean working practices Level of Independently testable and deployable software Low High Low High Slow Fast Continuous Delivery Daily/Weekly Independent Monthly Coordinated Quarterly Enterprise TEAM TECHNOLOGY DevOps Large: The Team Ceiling Do you have the team in place to excel? Designed for Continuous Delivery? • Training • Skills • Mindset • Goal alignment • Contracts • Empowerment No matter how good your technology ensure your team is up to it!
- 16. jmwpro The team’s level of agile/lean working practices Level of Independently testable and deployable software Low High Low High Slow Fast Continuous Delivery Daily/Weekly Independent Monthly Coordinated Quarterly Enterprise TEAM TECHNOLOGY Team Policy based on DevOps Do you have the team in place to excel? Designed for Continuous Delivery? • Training • Skills • Mindset • Goal alignment • Contracts • Empowerment No matter how good your technology ensure your team is up to it! INVEST
- 17. jmwpro The team’s level of agile/lean working practices Level of Independently testable and deployable software Low High Low High Slow Fast Continuous Delivery Daily/Weekly Independent Monthly Coordinated Quarterly Enterprise TEAM TECHNOLOGY DevOps Large: The Technology Ceiling Can you really do Continuous Delivery with your current architecture and systems? Designed for Continuous Delivery? • Vendor systems from the 90s • No unit testing • Slow/No build automation • Monolithic code bases/systems • Poor monitoring / logging No matter how good your team is your technology sucks!
- 18. jmwpro The team’s level of agile/lean working practices Level of Independently testable and deployable software Low High Low High Slow Fast Continuous Delivery Daily/Weekly Independent Monthly Coordinated Quarterly Enterprise TEAM TECHNOLOGY Procurement strategy based on DevOps Can you really do Continuous Delivery with your current architecture and systems? Designed for Continuous Delivery? • Vendor systems from the 90s • No unit testing • Slow/No build automation • Monolithic code bases/systems • Poor monitoring / logging No matter how good your team is your technology sucks! INVEST
- 19. jmwpro DevOps Small <aka DevOps Fun/>
- 20. jmwpro • Selling in over 100 countries. Flagship store in Covent Garden • Hand made in Great Britain in our own factory. • Opportunity to re-invent the technology, teams and ways of working • Autumn 2014 - Website in urgent need of a re-platform • DevOps and BDD fundamental to the way we work and technology strategy DevOps Small: Cambridge Satchel Founded 2008 by Julie Deane OBE, 120 Staff £600 initial investment in 2008 has led to £10M+ total revenues
- 21. jmwpro What if…. <clean sheet of paper/>
- 22. jmwpro The team’s level of agile/lean working practices Level of Independently testable and deployable software Low High Low High Slow Fast Continuous Delivery Daily/Weekly Independent Monthly Coordinated Quarterly Enterprise TEAM TECHNOLOGY App Strategy based on DevOps Clean sheet of paper. Design and Build Team and Technology from the ground up with DevOps in mind. Designed for Continuous Delivery: • End to End Test Automation • Build and Deploy Automation • DevOps culture • Code Reviews • API focus / API Mocks • Behaviour Driven Development • Cloud Dev Tooling • Cloud Infrastructure • Software API Web App
- 23. jmwpro Cambridge Satchel 2014 28th November Black Friday , “Can’t be done!” ;-) arrive Q: How do you develop, test & deploy software? • Tell me about DevOps and BDD? • How much of my budget is for automation? • Will I be left with legacy? Where’s the code? • Is my architecture decoupled? Crazy!
- 24. jmwpro Cambridge Satchel – dream team please! 2014 28th November Black Friday 20% Service Integrator Developers 20% Internal Engineers 20% API & Middleware Developers 20% BDD & Exploratory Testing 20% DevOps/AWS Skills 35K in 2013
- 25. jmwpro Cambridge Satchel 2014 28th November Black Friday launch 35K in 2013 87K in 2014 On new platform WE DID IT! Cambridge Satchel nominated for Demandware Pacesetter Award
- 26. jmwpro Some tips <thoughts on how to start/>
- 27. jmwpro Which needs to be fast moving? Which are tied to corporate release cycles Which have automated tests? Define & decouple your ‘Pace Layers’
- 28. jmwpro The team’s level of agile/lean working practices Level of Independently testable and deployable software Low High Low High Slow Fast Continuous Delivery Daily/Weekly Independent Monthly Coordinated Quarterly Enterprise TEAM TECHNOLOGY Fight the right battles with your legacy Where you invest your $$ is critical. Invest in DevOps where it matters. You can always start somewhere DevOps without legacy is easy.Front End UI Finance Systems Payment Order Mgt Core Ecomm Digital Asset Cust. Mgt Apps
- 29. jmwpro The team’s level of agile/lean working practices Level of Independently testable and deployable software Low High Low High Slow Fast Continuous Delivery Daily/Weekly Independent Monthly Coordinated Quarterly Enterprise TEAM TECHNOLOGY Enterprise equilibrium tends to push your DevOps adoption backwards Watch out for corporate equilibrium Make the wrong choice and the forces may be working against your goal of faster delivery. Wrong technology choice Wrong hiring policy Wrong contractual & financial frameworks Wrong 3rd Party Suppliers Wrong team objectives & rewards
- 30. jmwpro The team’s level of agile/lean working practices Level of Independently testable and deployable software Low High Low High Slow Fast Continuous Delivery Daily/Weekly Independent Monthly Coordinated Quarterly Enterprise TEAM TECHNOLOGY Fight the right battles with your legacy Where you invest your $$ is critical. Invest in DevOps where it matters. You can always start somewhere DevOps without legacy is easy.Front End UI Finance Systems Payment Order Mgt Core Ecomm Digital Asset Cust. Mgt Apps
- 31. jmwpro The team’s level of agile/lean working practices Level of Independently testable and deployable software Low High Low High Slow Fast Continuous Delivery Daily/Weekly Independent Monthly Coordinated Quarterly Enterprise TEAM TECHNOLOGY Reduce size and complexity of slow moving applications Reduce your legacy and create new capability Build New initiatives the right way E.g. consider creating a Front End separation layer enabling parts to be independently released NEW Legacy Zone
- 32. jmwpro The team’s level of agile/lean working practices Level of Independently testable and deployable software Low High Low High Slow Fast Continuous Delivery Daily/Weekly Independent Monthly Coordinated Quarterly Enterprise TEAM TECHNOLOGY Put all interactions between applications through APIs Understand the pace layers of your apps and governance needed How good are the major vendor Ecommerce and Finance/ERP systems? Kill Dependencies. Focus on APIs Front End UI Finance Systems Payment Order Mgt Core Ecomm Digital Asset Cust. Mgt Apps API API API API API API API API Centralise your API access (ie Mulesoft) BDD is great for defining and testing APIs as well as Uis
- 33. jmwpro The team’s level of agile/lean working practices Level of Independently testable and deployable software Low High Low High Slow Fast Continuous Delivery Daily/Weekly Independent Monthly Coordinated Quarterly Enterprise TEAM TECHNOLOGY When a new initiative comes along and a new team is built to deliver it set the bar high with DevOps operational requirements and ways of working. Encompass: • Behaviour Driven Development • Continuous Integration • Continuous Delivery • Full automation • Robust configuration management Set the bar high for new initiatives / programmes NEW Legacy Zone Don’t create new ‘legacy’
- 34. jmwpro The team’s level of agile/lean working practices Level of Independently testable and deployable software Low High Low High Slow Fast Continuous Delivery Daily/Weekly Independent Monthly Coordinated Quarterly Enterprise TEAM TECHNOLOGY API Shadowy IT, API Economy and DevOps Accelerating Revenue opportunities by providing robust and agile data services needed throughout the business. Marketing, Retail and Wholesale teams able to self-serve data and own their own integration with relevant partners delivered with external teams. Web App Finance My Team Everyone else…
- 35. jmwpro #jmwpro My blog, these slides and other musings available at: Thanks for listening
Editor's Notes
- #2: Intro: Firstly thank you for taking the time out to come and listen to me share some of our experiences of software at scale My name is Jonny Wooldridge, I am CTO of Cambridge Satchel company
- #3: WebMaster – that really was DevOps optimised in terms of alignment – Dev & Ops. Hard Devops inc Enterprise DevOps blog.
- #5: The control of the software delivery lifecycle and via Devops principles IMHO kept the programme on the rails. 650 Developers in multiple disparate locations – waterfall contracts. DevOps kept the project on the rails and honest. At the time there was nowhere really to go. None of our suppliers had heard of DevOps, nobody in IT. We had to define what it meant and why it was important. Project was iterative waterfall with outsourced and separate Development, Test, Support & Operations teams. No access to hardware. Not a particularly fertile ground for DevOps. The In-house engineering team were at the frontier of large scale DevOps and learnt an awful lot along the way. Overcame much more than just technology challenges.
- #6: On the spot – define what you do. The majority of us come to work to support the creation of code
- #7: The control of the software delivery lifecycle and via Devops principles IMHO kept the programme on the rails. 650 Developers in multiple disparate locations – waterfall contracts. DevOps kept the project on the rails and honest. At the time there was nowhere really to go. None of our suppliers had heard of DevOps, nobody in IT. We had to define what it meant and why it was important. Project was iterative waterfall with outsourced and separate Development, Test, Support & Operations teams. No access to hardware. Not a particularly fertile ground for DevOps. The In-house engineering team were at the frontier of large scale DevOps and learnt an awful lot along the way. Overcame much more than just technology challenges.
- #8: The control of the software delivery lifecycle and via Devops principles IMHO kept the programme on the rails. 650 Developers in multiple disparate locations – waterfall contracts. DevOps kept the project on the rails and honest. At the time there was nowhere really to go. None of our suppliers had heard of DevOps, nobody in IT. We had to define what it meant and why it was important. Project was iterative waterfall with outsourced and separate Development, Test, Support & Operations teams. No access to hardware. Not a particularly fertile ground for DevOps. The In-house engineering team were at the frontier of large scale DevOps and learnt an awful lot along the way. Overcame much more than just technology challenges.
- #9: The control of the software delivery lifecycle and via Devops principles IMHO kept the programme on the rails. 650 Developers in multiple disparate locations – waterfall contracts. DevOps kept the project on the rails and honest. At the time there was nowhere really to go. None of our suppliers had heard of DevOps, nobody in IT. We had to define what it meant and why it was important. Project was iterative waterfall with outsourced and separate Development, Test, Support & Operations teams. No access to hardware. Not a particularly fertile ground for DevOps. The In-house engineering team were at the frontier of large scale DevOps and learnt an awful lot along the way. Overcame much more than just technology challenges.
- #11: INTO to the diagram……. EUREKA MOMENT. The Team Does the team have the necessary skills, working behaviours and objectives to be able to deliver small increments of software regularly. Is the team the right team? Encompasses people and process. I can only think in 2 dimensions – a great team will have great team processes The Software Is the software the team is working suitable for fast delivery and has automated Build/Test/Deploy scripts? Most large scale ecommerce platforms were built before DevOps considerations were around. – great technology with have great processes. Of course / inherited / designing
- #12: From working on several initiatives within enterprises large and small it is evident that the level of DevOps maturity is driven by two factors: The Team Does the team have the necessary skills, working behaviours and objectives to be able to deliver small increments of software regularly. Is the team the right team? Encompasses people and process. I can only think in 2 dimensions – a great team will have great team processes The Software Is the software the team is working on / inherited / designing suitable for fast delivery and has automated Build/Test/Deploy scripts? Most large scale ecommerce platforms were built before DevOps considerations were around. – great technology with have great processes.
- #13: From working on several initiatives within enterprises large and small it is evident that the level of DevOps maturity is driven by two factors: The Team Does the team have the necessary skills, working behaviours and objectives to be able to deliver small increments of software regularly. Is the team the right team? The Software Is the software the team is working on / inherited / designing suitable for fast delivery and has automated Build/Test/Deploy scripts? Most large scale ecommerce platforms were built before DevOps considerations were around.
- #14: From working on several initiatives within enterprises large and small it is evident that the level of DevOps maturity is driven by two factors:
- #15: INTO to the diagram……. The Team Does the team have the necessary skills, working behaviours and objectives to be able to deliver small increments of software regularly. Is the team the right team? Encompasses people and process. I can only think in 2 dimensions – a great team will have great team processes The Software Is the software the team is working on / inherited / designing suitable for fast delivery and has automated Build/Test/Deploy scripts? Most large scale ecommerce platforms were built before DevOps considerations were around. – great technology with have great processes.
- #17: Decoupling and
- #19: Decoupling and
- #21: Julie Deane started in her kitchen Every technology decision being made is now with a backdrop of DevOps and BDD Now back in start-up world at Cambridge Satchel but the enterprise lessons are key to building a successful and relevant technology strategy which has longevity and agility.
- #23: Decoupling and
- #24: Questions to SI? Will run the project myself and bring in other key players.
- #25: SI didn’t have a clue about DevOps or BDD. What if I built a dream team.
- #26: Questions to SI? How do you develop software? Will I be left with legacy? Is my architecture decoupled yet connected? Will run the project myself and bring in other key players.
- #28: Don’t boil the ocean – Danger: Don’t assume they all go at the same speed! Define you path to production which will help you…. Define what paces you have FAST - applications you can continuously deliver that are decoupled and have a great % of automated tests MEDIUM – applications that are coupled and need end to end regression testing (hopefully mostly automated) SLOW – Projects dependent on Corporate Release Cycle (SAP etc.) Put in place different governance for different paces Understand your pace dependencies
- #29: You need to make a decision early on where you want to focus your energies. Many legacy or corporate applications have been outsourced and might have a slow delivery cycle and governance model but this might be appropriate for the application in question. Aim High: Aim to have continuously delivery capability for all your high priority applications. HOWEVER, if there are reasons why you can’t or there are constraints, make sure you can continuously deliver your apps into lower environments. Factors influencing how far you go with Continuous Delivery: How many environments you have. Suitability of software and the amount of automated tests. How well you have control over the deployments made into those environments. How many other teams are sharing the environment for their projects
- #30: Hosting partners – no access to code. The reason being, the technology and staffing choices you make now will affect how you can scale in the afuture. Choices that in the past might have been judged on a single project delivery are now judged on capability to deliver incrementally ongoing.
- #31: You need to make a decision early on where you want to focus your energies. Many legacy or corporate applications have been outsourced and might have a slow delivery cycle and governance model but this might be appropriate for the application in question. Aim High: Aim to have continuously delivery capability for all your high priority applications. HOWEVER, if there are reasons why you can’t or there are constraints, make sure you can continuously deliver your apps into lower environments. Factors influencing how far you go with Continuous Delivery: How many environments you have. Suitability of software and the amount of automated tests. How well you have control over the deployments made into those environments. How many other teams are sharing the environment for their projects
- #32: Rather than try to take an existing application and continuously deploy it, those parts of the application that will need to change frequently to be independently deployable. As an example keep the majority of the application the same but create a Front End architecture that can be ‘published’ on demand without a major release of core services.
- #33: Back to the paces – let’s put some colour (literally on this)…. Many systems are difficult to release fast due to lack of test automation coverage (of all types). Some Core Ecommerce apps sold by the major vendors are too difficult to build, test and deploy to allow for fast test cycles and frequent releases.
- #34: How many people have new projects that you’ll be undertaking. Raise the bar high, do things differently. You have the advantage of widespread understanding and proof that DevOps makes a difference. What’s stopping you doing the right thing on greenfield projects?
- #35: Decoupling and