Making quality visible in Product Engineering

Editor's Notes

• #2: QA are all the activities we do to ensure correct quality during development of new products
• #3: What we in SuperOffice wanted to achieve was to find the defects early in the development cycle, preferably before they where implementedfinding defects should be the exception not the rule. if verification triggers test- and fix cycles, then the dev.process is defective.What you want Is a culture where you don't blame testers for escaped bugsIn the earlier days, Quality Aassurance was initially used, like in World War II when munitions were inspected and tested for defects AFTER they were made. Today's quality assurance systems emphasize catching defects BEFORE they get into the final product and by that eliminate waste
• #5: I strongly believe in these three principles
• #6: Software product development is about making a great product that your audience want to buy. You do this by building the product first and then people will evaluate it and buy it if they like it."Like it" does not only mean functionality, but also that ithas an easy to understand designactually works as intended, meaning that the user gets a good feeling of the product in his experience of using it
• #7: Quality is important for good user experience. Not only in the design, but also in the materials usedYou wouldn't build a great design chair using rotten wood, would you? Steve Jobs was extreme in his thoughts about this when he built his Macintosh computers. You don't need to go that far, but it is important that:Good quality architecture and code that is maintainable AND extensible over time/versions
• #8: You want to eliminate waste because you need that time to work with your next versionmore bugs means more time until you are able to releasefor every month you delay your release because of bad quality (it is not ready) you both loose sales and the opportunity to start on the next version of your product= bugs are waste
• #10: This might look like a finished project ready to ship, but is not! I believe in what I see and not what project plans tells me. I need to see a working piece of software to know the statusThe picture above is from early 2000. The cards on the wall are not user stories, but functional areas (large piece of functionality)In this image there are a lot of hidden work – bugs, not completed features, works on my machine etc= You do not know the status of this project before you have tested it
• #11: In theproject:Eachbugfound late in thedevelopmentcycleinvolvesseveralpeople and administrationofthebug; Tester (findsbug, logs it)Product Owner (reproduces thebug, evaluatesthebug)Developer (reproduces thebug, fixes it, developers test)Tester (verifiesfix)This sums up to manyhours used pr bug = waste
• #12: After the Release:In Product Development you must always think of your next version and start developing it as soon you have released. You don't want to use your time fixing faults in a released product ==> It ruins your flowWhen you release you should know the defects that exist in your product. Your software will always have errors, but you should know about them so you are able to evaluate which ones to fix and which ones to ship with your softwareSpending less resources on older versions means you can focus on improving your product-line and delivering a better user experience.
• #13: I believe that taking QA seriously, doing things right the first time will free up time for improving your product and make it more competitive
• #14: Not time to tell abouteverythinghere, but I willgiveyou 5 thingsthatwe have done to ensurethat QA is visible in our R&D department and is a part oftheeverydaylifeofeach and oneofus, not onlythe testers.
• #16: This vision actually sets the standardIf you are so lucky that your company have a vision, you can find a lot of useful information here. The SuperOffice vision guides us in where to put our efforts when we develop our software. We can spend some time on a user control if we think it will apply with our vision about being more efficient and easier to use.The same goes for bugs – our software should POSITIVELY influence our users, and you don’t do that with a piece of software with serious defects OR a lot of minor defects that the user runs into all the time => gives a negative user experience and not positive
• #17: At one time in SuperOffice we started to talk about how we really wanted to develop software. This was important because it created a concensus for the basic To establish the culture we started with study groups. Once a week at almost at the end of the day, we gathered in a study group and discussed books like “Code Complete” and “Clean Code” (Steve McConnel), Design Patterns, The Deadline, Peopleware (Tom DeMarco) to mention a few.
• #18: The code must work on the test servers – no discussion and it is your job to make sure that it worksThere is no such thing as a 10 min job: It is not just to "add" not planned functionality just because the developer sees an opportunity. It´s inevitable that it will involve other resources from Design, Testing, error handling and prioritization by Product Owner etc also = use a lot of time not scheduled forA hero in your team that always fixes your problems as a Project Manager, but very often with consequences revealed later.= this causes problems
• #19: How to build the QA culture - At SuperOffice we all have the title "Product Developer", but we have different skills in designing, development and testing etc.
• #20: Everybody participates – all meetings are rescheduled – it is Fun and we all learn new things about our product 
• #21: Reviews and retrospectivesareactivitiesthatreally supports theconceptofthat QA is everyonesresponsibility. Thesearetechniquesthatare used bothearly and late in thedevelopmentprocess and theyarethere to ensuregoodqualityearly or to improveyourprocess and learn from yourmistakes.
• #22: Testing in thiscontext is Reviews, Presentations ofhowwethinkaboutthatthenewarchitectureshould be (Technical review) etc…
• #23: Reviews are the maybe mot visible QA thing you do since it involves almost everyone. Shows that you have to deliver Quality before you send you work to the next person in the working chain.Sogeti claims that 40% of the bugs are introduced before the coding starts. This very much align with what Tom Gilb also saysStart early - look for bugs in your documentation. Use techniques like: Document review, backlog meetings, 1/3 presentation etc. This involves Product Owners, Designers, Testers and Developers and makes the Quality Assurance aspect very visible
• #24: Even if you do SCRUM and sprint retrospectives, it is also a good idea to run a project retrospective and get an analyze of the whole project from start to end.The Project Retrospective technique does not focus on blame, but that we did what we did because of the situation at hand and the knowledge we had at that time
• #25: Within Product Engineering it will be the same peoplethatwillworkonthenextversionofyourproduct. It will be veryuseful to analyzewhatworked and whatdidwe not yetknowhow to dealwith etc.Whathappened in theprojectWhydid it happenWhatcanwe do to make sure it does not happenagainThe workshop aims to build 5 posters:Whatworkedwellthatwewill not forgetWhat have welearnedWhatshallwe do differentlynext timeWhatweneed to discussfurther, still puzzles us.Whatwedon’tknowhow to solve at this moment
• #27: You want to have everybody on boardPeople are only able to adjust to a few changes in each project, not all at once
• #29: QA is very visible in the Agile and SCRUM process.Sprint TestPerformed by a QA person when a part of a functionality is finishedimplemented to ensurethat it works and that it workswithother parts ofthefunctionality and the rest oftheapplication.Functional TestPerformed by a QA team when a functionlity is finishedimplemented and all bugsfound by Sprint Tests arefixed. Last test offunctionalitybefore a System Test.
• #30: Tools, like TFS - everything is in the same tool. Apply rulesUsing a tool that contains functionality or is integrated with all sub-systems so that everybody easily gets an overview of open User Stories and unresolved bugs at any time makes both the progress and quality more visible for every team memberAbilty to apply rules for submitting of code
• #31: This is where you connect with your users
• #32: We are often working “behind closed doors”, only our bosses gets to meet our customers and users now and then
• #34: This is also an opportunity for you to let your Product Developers experience how the product they have created is received by your usersIt is easier to understand the problem of a user if you get to see what a fault in your program causes him in his daily work.Gamification – possible for users to “like” functionality so that your dev.teams get scores?In two projects we have used a pre-beta program with 20 users that have helped us testing. They where given a short test assignment that also teached them about the new functionality. We shared the result in a tool where everyone could watch the progress and the comments written by the testers. We answered the testers ASAP and logged bugs if they found anything interesting – the best part; the users thought it was fun  Free testers 
• #37: …becauseyou do it right the first timeDon’tneed to use time on fighting bugs