QA and scrum

Editor's Notes

• #2: The name of the presentation that I have been asked to give is QA and SCRUM. This made very little sense to me and I hope that by the end of this talk some of you at least will look at this title and also think for a moment and also wonder if this was correct. So for my own sanity, in the meantime, I and going to rename this to "QA and Agile" and we can see if that works a little better.Now given the kinds of things I hear in most of the interviews that I do with prospective software developers, my guess is that some if not most of your are sitting there going "whatever SCRUM agile, same thing". If this is the case then keep listening, because I am about to give you some very important information, especially those of you who are going to be asked this in an interview at some point.
• #3: So to start off, lets put the elephant on the table and clear the air. "SCRUM is not agile" ... shock and horror.
• #4: How could I say something like that? If you will humour me, lets take a step back and take a look at what agility is in the world of software development. I used the word agility deliberately because sadly the word agile has been abused to the point that it has become meaningless. Seeing someone writing on their CV that they are proficient in Agile or that they use Agile above all is an attack on the English language, saying I am proficient in Agile is as sensical as saying I am proficient in pretty. It shows how we have lost the meaning of the word.Instead, we should be thinking of the word and using it as an adjective, so:You aren't an agile developer - you are a developer who programs with agilityYou don't use agile tools - you use tools that enhance your agilitySo, now that we have the English lecture behind us, lets look at what that means from a developers perspective.
• #5: The best way I have found to describe agility is using a model developed through a study done by Dianna Larson and James Shore called agile fluency. 'Agile fluency' is used to describe a teams evolution or progress by means of a "star" system where each star represents distinct stages in their evolution. Each star brings specific benefits as well as adoption challenges. At this stage it should be added that although there are 4 stars defined in the model, the 3rd and 4th star may not be applicable to all teams/companies and for the purposes of this presentation we are not even going to discuss them.One of the core mind-shifts required is that fluency is defined as the entire teams fluency and not an individual members, this 'team' concept manifests itself in every aspect of agile and we will see more and more how team cohesiveness and collaboration is fundamental to fluency. Don’t make the mistake of blaming individuals for low team fluency, or assuming that one highly-skilled individual will guarantee high team fluency. Becoming agile will require buy in from the entire team as well as senior management.Lets get into it.
• #6: One star fluency defines the management pillar of agile, the adoption of SCRUM would be an example of achieving 1 star fluency. This is the easiest fluency level to achieve and research has shown that it takes about 6 months to become fluent at this level and simply requires a team culture shift. Teams are introduced to the notion of a continually changing backlog as the list of requirements is groomed and re-prioritised according to business value.Short iterations are mandated to deliver small pieces of business value providing stakeholders the transparency as to what the team is working on and the progress, it also allows changes in direction if required rather than only getting sight of the finished product at the end only to find it was not what was expected.This level of fluency is pretty well understood by most, but sadly from my experience, many people would consider themselves as agile having achieved this first level of fluency. This is not the case, only having 1 star fluency is not enough to consider yourself agile. In fact I would go so far as to say that if this is as far as you are intending to go then you are better off not going the 'Agile' route.
• #7: Two star fluency is focussed on the technical aspects of software delivery, the adoption of Xtreme Programming techniques is required for achieving this. This is the most skill-intensive level to achieve and it has been shown to take up to 2 years and requires a team skill shift. TDD(Test Driven Development), BDD(Behaviour Driven Development), peer code reviews and pair programming are some fundamental techniques that developers need to come to terms with.Success of 2 star fluency is measured by the teams ability to ship the software at the drop of a hat. This would of course not be possible without the likes of a continuous integration environment that runs though all the automated test suits and builds the application artefacts every time any code is changed.I have in the past worked for a company that had a code freeze 6 weeks before a release. That worth rephrasing FOR 6 WEEKS BEFORE YOUR PRODUCTION RELEASE YOU ARE NOT ALLWED TO TOUCH THE CODEBASE....The flip side of that, I went to a conference of software scalability last year where one of the presentations was give by one of the senior engineers ate etsy.com and he said that this year one of their major focusses is to get the build time down from 15 minutes. The reason for this was that in a 10 hour day, a 15 minute build time (which includes building all the binaries and artefacts and running all the automated tests) means that they are only able to push out 40 production releases a day! and this was not acceptable for them! I don't suppose I need to point out which of those teams has evolved to 2 star fluency.
• #8: The thing about building a software system over time is that it accumulates a certain amount of technical debt as frameworks change or better mechanisms and patterns are developed. Keeping a codebase clean and maintainable is essential for any software with any kind of longevity. Code should be continuously refactored and this would be a highly risky behaviour without having a safety net of automated test suits that would be run. And over time as the codebase grows, without this safety net, you find yourself spending less and less time actually delivering business value and more time manually testing and hoping like hell that you haven't broken anything (which by the way is futile) until you get to the point where you have to lock down your codebase for weeks before releasing anything into production.
• #9: Automated tests have more benefits than the obvious testing they do.Have you ever integrated a third party library into your project? You got a big manual full of nice documentation. At the end there was a thin appendix of examples. Which of the two did you read? That's what the unit tests are! They are the most useful part of the documentation.They are the living examples of how to use the code.They are design documents that are hideously detailed, utterly unambiguous, so formal that they execute, and they cannot get out of sync with the production code.
• #10: There is a common understanding that the sooner you detect bugs the cheaper it is to fix them. Fixing bugs once they have been released into production is exponentially more expensive that having picked them up during development. So writing automated tests to run every time you commit code into your source code repository is the perfect solution is it not. For the benefit of all the Barry Roux fans... "I put it to you" that you can go one step further and write the tests before even beginning to write the production code.
• #11: For those of you that have not heard of Uncle Bob, do yourself a favour and google him and read his articles on writing quality software. He has 3 golden rules that he lives by and so should we as software professionals:1. You are not allowed to write any production code unless it is to make a failing unit test pass.2. You are not allowed to write any more of a unit test than is sufficient to fail; and compilation failures are failures.3. You are not allowed to write any more production code than is sufficient to pass the one failing unit test.
• #12: Why, you ask is this better than writing tests after the fact?In the words of uncle BOB himself, if you have ever tried to add unit tests to a system that was already working, you probably found that it wasn't much fun. You likely found that you either had to change portions of the design of the system, or cheat on the tests; Because the system you were trying to write tests for was not designed to be testable. For example, you'd like to test some function 'f'. However, 'f' calls another function that deletes a record from the database. In your test, you don't want the record deleted, but you don't have any way to stop it. The system wasn't designed to be tested.
• #13: When you follow the three rules of TDD, all your code will be testable by definition! And another word for "testable" is "decoupled". In order to test a module in isolation, you must decouple it. So TDD forces you to decouple modules. Indeed, if you follow the three rules, you will find yourself doing much more decoupling than you may be used to. This forces you to create better, less coupled, designs.
• #14: Automated tests come in a variety of flavours but can be broken into 2 categories: Unit tests, integration tests, smoke tests are known as white box tests or technology facing tests in that the developers have sight of the code that they are testing and the test that the code is written properly or the internal quality of the code written. Acceptance tests are black box tests or business facing tests in that they are written to test that the functionality delivers the correct output or put another way they test that the right code is written or external quality of the code. These tests do not need to be written by the software developers themselves.
• #15: Depending on how you interpreted the title of the presentation you could see all of the above as addressing the QA of software developed by an Agile software team... or maybe QA refers to a specific role in the development team. Lets explore that, software teams like the ones I described earlier with the 6 week lockdown also typically have "testers" who spend days following a test pack to ensure that the requirements of the new functionality have been implemented properly. Now in our new team that is delivering multiple production release per day, what becomes of this oldschool QA role. As I have just described, the acceptance tests are business facing and tests ensuring that the functionality is working as expected and these tests can be written by non-developers who have business insight and this role could easily be filled by the traditional old tester/QA role.
• #16: This requires a person to elaborate on the SCRUM stories that are being prioritised on the backlog by adding a list of acceptance criteria in the form of Given ... When... Then. These acceptance criteria are then use by the developer verbatim in their acceptance tests, and this concept is known as Behaviour Driven Development or BDD.
• #17: Here is an example of the acceptance criteria written that are used verbatim shown in my IDE that are used as input into my acceptance tests run by JBehave or cucumber