A Roadmap to Enterprise Quality
Download as PPTX, PDF1 like393 views
The document outlines the enterprise quality vision of Farm Credit Services of America, emphasizing the importance of a unified approach to quality in software development. It details the strategies and tools, such as a shift-left model, Lean Quality Delivery (LQD) goals, and testing automation through Azure DevOps and Jenkins, aimed at enhancing collaboration and efficiency. The impact of these initiatives has resulted in increased test automation, improved software delivery confidence, and a focus on continuous improvement in quality practices.
A Roadmap to Enterprise Quality
- 1. Road Map to Enterprise Quality Jeff Bramwell VP – Solutions Architecture Farm Credit Services of America jeff@moonspace.net @jbramwell 19 July 2019
- 2. Titanium Sponsors Platinum Sponsors Gold Sponsors
- 3. About Farm Credit Services of America • Serving farmers and ranchers in rural America • To be Agriculture’s Most Valued Financial Partner • ~$28 billion in loan volume (2018) • ~140 Applications Development teammates across 17 teams • ~110 Technology Services teammates • Primarily a .NET shop with JavaScript/Angular + Mobile tech.
- 5. Agenda • Enterprise Quality Vision – Why? • Strategy • Tooling
- 6. Enterprise Quality Vision • Why?
- 7. It is not the strongest of species that survives, nor the most intelligent, but the one most responsive to change. - One interpretation of Charles Darwin’s writings – “On the Origin of Species”
- 8. Why Enterprise Quality? 200+ Applications & Services Growing Customer Base 17 AppDev teams Digital Transformation 1,500+ deployments in 2018
- 9. Why Enterprise Quality? Deliver More Value, More quickly!
- 10. Strategy • Shift Left • People • Process • Tools
- 11. Shifting Left
- 12. Success Takes People! • Quality Assurance role – not just for testing any more, also… • System health analysis • Visibility & understanding • Peer group • Collaboration • Development • Ability to understand and write code
- 13. Focus on Enterprise Quality • Solutions/Enterprise Architect with focus on enterprise quality… • …and health • …and visibility • …and metrics/KPIs • …and consistency
- 14. QA Role QAEQAD
- 15. Lean Quality Delivery (LQD) • Vision • To Be the Chosen Partner • Goals • Functional quality – is it doing what it’s supposed to do? • Structural quality – is the code testable, secure, maintainable, etc.? • Process quality – are the software development processes efficient and effective? • Strategies • Consistent across teams • Tactics • Can vary by team and project • Teams revisit LQD/tactics at the beginning/end of projects
- 16. Lean Quality Delivery (LQD) – Cont’d Lean Quality Delivery (LQD) Vision Goal Strategy Tactics Score (1-5) ToBetheChosenPartner FunctionalQuality 1. Delivers the desired value Story writing and review UI automation testing Performance testing (Does it meet SLA needs?) Service-level testing Unit testing Discussion with development teams 2. Deliberate testing plan at the story and project level Testing starts with the creation of the story and review Testing is a first-class citizen Tests should answer questions o What makes the story done? o How is the user going to use it? 3. High reliability, minimal defects Pairing/Code reviews Health monitoring (Are we notified when something goes awry?) 4. Delightful user experience User interviews (in our case, this is typically developers) Process mapping Prototyping Follow-up discussions/surveys StructuralQuality 1. Design for testability Incorporate design patterns (as appropriate) TDD/Code reviews Continuous Integration Continuous Delivery 2. Security best practices Security training for developers Static and dynamic code scanning (for public-/partner-facing apps) 3. Follows project guidance Application reviews Reuse of enterprise components Reference Tech Radar (and follow the process for new technologies) Create Project Start Architecture document ProcessQuality 1. Responsive support Proactive support (e.g. Do we send “ack” within 15 minutes?) Someone takes ownership of issue and “walks it through” 2. Consistent Process The team can describe the process and how it impacts quality The team implements the agreed upon tactics Reviewing and grading of the team this document 3. Predictable Delivery Measure Cycle Time Groom Backlog Manage Budget Manage Timeline Automated deployments 4. Visibility & Transparency of Count defects and kickbacks Count production-related issues System Health (for systems under our team’s control) 5. Routinely follows process for feedback and learning Retrospectives Demos 5 Why's for production defects
- 17. Lean Quality Delivery (LQD) – Cont’d
- 18. Process – CI/CD
- 19. CD vs. CD • Automation is key • But which is it – Continuous Delivery or Continuous Deployment?See the difference?
- 20. CD vs. CD – Cont’d • What’s the difference? • Trust! • It will take time I don’t trust words, I trust actions.
- 21. Tooling
- 22. Version Control • Use version control as the single source of truth for Automation Scripts. Dev • Azure DevOps • Visual Studio • VS Code • GIT • Common lib • Project Repo QA • Azure DevOps • Visual Studio • VS Code • GIT (SourceTree) • QA lib • QA Repo Software Development/Quality Tools
- 23. Communication Dev • Lead Dev Meeting • AppDev Team Blog • SharePoint • Microsoft Teams • Email Distribution Lists • DOUG QA • QAD Meeting • AppDev Team Blog • Microsoft Teams • QA Website • QA SharePoint • Email distribution lists Communication as a Tool
- 24. Learning Dev •On-site Training •Dev Pairing •Pluralsight •Website/Blogs •Tech Conferences •DOUG •.NET/Other User Groups QA •On-site Training •QAD/Dev Pairing •Pluralsight •Website/Blogs •Tech Conferences •Book Club •DOUG Learning Opportunities
- 25. Applied Testing (Testing Pyramid) UI Telerik Cypress Manual Integration FitNesse Postman jMeter Fortify (Security) Unit NUnit Jasmine/Karma FakeItEasy NCrunch, Wallaby.js Visibility via Jenkins & Azure DevOps
- 26. Jenkins • Why Jenkins? • Flexibility! • Single Platform • Highly configurable • Automates the build and test (we use it for scheduling/running tests) • Get source code from repository (Azure DevOps) • Dashboard - everyone can see what’s happening • Generate report and notify stakeholders of build status • Plug-in extensibility: Unit test coverage, last success and failure, trends
- 28. Azure DevOps • Why Azure DevOps? • Integration with development, build & deployment (CI/CD) processes • Familiar platform to developers • Highly configurable • Automates the build, test and deployments • Get source code from repository (Azure DevOps) • Dashboard - everyone can see what’s happening • Plug-in extensibility: Unit test coverage, last success and failure, trends
- 29. Azure DevOps Example Dashboard
- 30. Azure DevOps Example Dashboard
- 31. QA VM Image Target Install • Telerik Runtime/ Chrome Plugin • Postman(Node/Newman) • jMeter • NUnit Common Software • GIT • Jenkins • Beyond Compare • Notepad++ Group Policies • Auto-login with test account, Disable screen lock, UAC • Browser settings, Screen resolution Base Windows 10 Image • With standard configuration
- 32. Summary • The problem: No unified approach to quality • Solution: • Adopted shift-left approach to software development and quality • Implemented the testing pyramid • Increased collaboration and communication between QA and Devs • Testing architecture using Jenkins and Azure DevOps • In other words… consistency • Impact: • Friction leads to lower adoption rates • We increased test automation from 400 tests suites per day 2,300 • Standardized on a common set of tools • Increased collaboration across teams • Increased confidence in delivering software
- 33. What’s next ? • Continue to focus on enterprise health & visibility • Further grow our shift-left practices • Automate our test data scenarios • Grow the quality community • Deliver great, high-quality software and experiences… faster!
Editor's Notes
- #3: We can’t do this without the sponsors! Thanks, sponsors!
- #5: Let’s have a dialog! We can all learn from each other.
- #9: While we are not to the same scale as the “Microsofts” or “Googles” of the world, we have been increasing our # of deployments dramatically year over year. As we move into more of a micro-/mini-service architecture mindset, it is expected that our number of deployments will increase more quickly than ever before. $28.4 billion in loan volume in 2018 up from $22.1 billion in loan volume in 2014
- #10: No matter how much value you deliver, the number of requirements you implement, the amount of functionality you build into your products, your business users and customers will always want more. If it’s not new features in your existing products, it’s new products that you’ve yet to create.
- #11: Provide leadership and help bring focus and visibility B2B Strategy: Basement living to Balcony Quality has been an after thought, treat it as a first-class citizen
- #12: Image Source: https://xebia.com/blog/its-2017-test-automation-is-not-optional-when-building-mobile-apps/ Get QA involved up front – during the initial stages Test earlier in the process
- #15: We shifted our Quality Assurance Engineer (QAE) role to a Quality Assurance Developer (QAD) role. This decision was made to help our transition to a shift-left mindset.
- #17: Provides a consistent mechanism for all software development teams to apply focus toward quality.
- #18: Another view of the same LQD from the previous page.
- #19: Are you collaborating with your QA(D)? Are quality efforts discussed as part of your project planning? Continuous Integration (CI) is a development practice that requires developers to integrate code into a shared repository several times a day. Each check-in is then verified by an automated build, allowing teams to detect problems early. Continuous Delivery (CD) is a software development discipline where you build software in such a way that the software can be released to production at any time - http://martinfowler.com/bliki/ContinuousDelivery.html Continuous Deployment means that you are engaging in actual deployment.
- #20: Continuous Delivery is a great milestone! However, there is more to be achieved by taking the next step toward Continuous Deployment.
- #28: “Demo-ware”
- #30: “Demo-ware”
- #31: “Demo-ware”
- #33: There was no unified approach in our Quality Practices(tooling and process) across our 10 AppDev teams. We were good at Continuous Integration but inefficient at Continuous Delivery.