Sourav_Kumar_SKUM279_Manoj_HYD_My Journey as a Software Testing Professional from Waterfall to the AGILE World
- 1. 1 TestNext Symposium 2016 My Journey as a Software Testing Professional from Waterfall to the Agile World Sourav Kumar Email id:Sourav.kumar@optum.com UnitedHealth Group HITECH CITY 2 BUILDING No.H09, GACHIBOWLI VILLAGE, TS, INDIA-500081
- 2. 2 ABSTRACT: We will find multiple articles or definitions online about the waterfall and the Agile model but we seldom get the opportunity to learn from the experiences of a software testing professional about the challenges he faced and the experiences he had while changing from Waterfall model to Agile in projects. This paper discusses in detail the experience of a software testing professional in UHG while the Organization was changing its delivery model by moving from Waterfall to the Agile World. This paper includes the challenges that the software professional faced initially when he was reluctant and opposed to the change and the advantages he sees at the end when he started working completely in agile model. This paper will try to explain in detail how difficult it was to move from waterfall to the agile mindset for the professional and will try to narrate the entire experience gained while working in a project over the period of two years .The project that will be discussed started off with 100% waterfall model, gradually moved to hybrid (waterfall -agile) model and eventually when it was a 100% agile model. Once it was pure agile, it became easy to work and deliver quality products in short span of time with highest user satisfaction. It will also illustrate how difficult it was to work with continuously changing requirements, how to give estimates on the go and complete the work in an environment where every minute was worth in gold and was being tracked. This paper will be limited to the views of software professional and it might not comply with all professionals, however the paper is written with a positive mindset keeping in mind the benefit of entire target audience who will be part of it. INTRODUCTION: Before I go ahead and narrate my experiences of moving from waterfall world to the agile world,its important for me to explain clearly what does waterfall and Agile actually mean.This paper is not limited to audiences who are familiar to Agile and Waterfall model but also to everyone else who have not yet heard of these models in the Testing Community.In order to benefit all readers let me start off with a very brief introduction of these two models followed by the detailed answers to the questions that actually occurs to every software testing professional when they move from the waterfall to the agile world. What is AGILE model? Fig 1: Agile Model in Detail
- 3. 3 Agile is a concept or methodology that involves iterative development in short cycles. It includes a high level of interaction with users or business representative, and its flexible response to change is a positive for both business teams and project teams. Agile development is based on a concept known as the Agile Manifesto, which is: We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: Individuals and interactions over processes and tools, working software over comprehensive documentation, Customer collaboration over contract negotiation, Responding to change over following a plan. For a better understanding of entire steps followed in an agile model, see the fig above. What is WATERFALL model? Fig 2: Waterfall Model in Detail Waterfall model is a process that involves stages that are run from start to finish before commencing the next stage. These stages are generally classified as: 1. Analysis - a preliminary analysis, proposes alternative solutions, describe costs and benefits and submit a preliminary plan with recommendations. 2. Requirements – gathering the requirements of the system and what needs to be developed 3. Design – designing the system 4. Implementation – developing the software system 5. Verification – testing the system, both system testing and user testing 6. Maintenance – ongoing support or maintenance of the system In this methodology, the requirements are gathered at the start of the project, with the aim of confirming them at the start to reduce cost and time impact later in the project. It is a common belief that finding discrepancies and issues earlier in the project will result in less time being spent than if they were found later in the project.Waterfall is the more traditional method of development. It was the main methodology that was taught during my college and trainings and even in the initial years of my working in IT Organization. Waterfall vs Agile One of the biggest discussions in IT project teams lately is the discussion of can we switch to agile methodology from the existing software development methodology for some of the projects? If this has left you also a bit confused, or if you just want to know what the differences of agile vs waterfall are, or if you want to know the answer of some of the generic questions that come up during the transition from waterfall to the agile, please read on.
- 4. 4 With the help of this Paper I have tried to come up with the answers for the below set of questions which play a very important role in understanding the minute details needed for transition from Waterfall to the Agile world for a software professional. 1) How a Software professional will be able to convince oneself to build everything but with limited time, limited resources and ever changing market? 2) How tools like ALM used mostly in waterfall was replaced with Rally as a single consolidated tool for all testing activities like test cases, execution report, defects logging and pulling all reports that were pulled using ALM earlier? 3) How this paper will help understand the basic similarities and differences between the Waterfall and Agile model and will help to achieve quality work in future with a model suitable for their project? 4) Why it’s always a win –win situation with Agile compared to waterfall model? 5) Real Life Examples /Case Study/Images illustrating the two models and their challenges/benefits. Keeping above questions in mind, let me try and answer them in an easy to understand manner and based out of my experiences. Fig 3: Waterfall vs Agile 1) How a Software professional will be able to convince oneself to build/test everything and learn alongside but with limited time, limited resources and ever changing market when his model is changed from Waterfall to the agile one? Software professionals can build/test everything and learn alongside with limited time, limited resources and ever changing market by applying the continuous, fast learning and delivering technique followed in agile. What I experienced in my initial days of Agile methodology was when I was encountering on a daily basis with so many Agile terms as shown in the image below and my first challenge was able to understand each of them and their relationship with each other. If you really want to succeed in Agile, make sure that you are really familiar with all fundamentals of Agile and rally tool. Since I was new to this methodology, I had lot of challenges but eventually with lot of hard work I was able to learn fast.
- 5. 5 Fig 4: Commonly used terms in Agile To keep pace and able to deliver quality products to my end users/customers, it was very important for me to follow below rules of agile. Fail fast but Learn Quickly. Fail Often but Learn continually. Fail Cheap but Learn inexpensively. I was eventually feeling happier to see the amount of work I was getting done at the end of each day in this new model .Within the first six months I noticed a big change in the pattern of my work style and delivery. I could also see that initially my project was operating in a waterfall model and then slowly we started operating in hybrid (waterfall-agile) model where team was operating partially in waterfall because they were unable to adapt fully to the new model. On this day when we are fully operating in an agile model I could easily make out below points which were contributing for a better productivity and better quality while working in Agile: a. Documentation reduced significantly for me as well and I was able to focus more on my actual tasks leading to more productivity. b. All documents/requirements were present in one single source Rally and attached to the particular user story/requirement I need to work on which saved a lot of time which used to get wasted in searching the right version of a document. c. My interaction with Business/stakeholders increased due to the demo/acceptance of my work scheduled for each user story I worked on. d. All my test activities progress and status were also tracked through single rally tool. e. Based on the progress made each day, we were able to identify whether the team is on track to complete the remaining work in the sprint. f. With each defect logged, we were able to add the hours to the defect and showcase why the project got delayed due to the delay in solving the no of defects present in the release along with time consumed for fixing the same. g. Multiple Reports could be generated on a daily basis to monitor progress on sprint level, release level, team level or project level. h. For any changes done to any user stories/tasks, defects, test cases notifications are present within rally to track the same if needed. i. Lastly if needed we have the option to sync all defects from rally to ALM in a particular instance during the initial phases of transitioning. j. Only in some cases, where we felt that a user story could have been well documented like it was in case of waterfall model. In those cases we have to go back to business and ask them to attach/update the user story with sufficient details so that we can proceed working on it independently. k. I was working more closely with all team members and the work style was very much transparent.
- 6. 6 2) How tools like ALM/QC used mostly in waterfall was replaced with Rally as a single consolidated tool for all testing activities like test cases creation, execution report, defects logging and pulling all reports that were pulled using ALM earlier? This will be the most important answer for the testing community as most of the traditional testers use ALM/QC for all testing related activities and even the managers for pulling various metrics/graphs from it.I will try to illustrate through various images the same functionalities being present in rally tool and can be used to replace ALM ,an overhead tool to manage the project .In case of rally we can have everything from requirement to user acceptance at one place rather than maintaining traditional multiple documents for many phases of testing .Considering the limitation of this paper ,I would just illustrate/compare the basic features like Requirments ,Defects andd Reports Generation of ALM and Rally through images/figs .While I would give the image for both but would be explaining only Rally considering most of the testing community is aware of ALM as it’s the traditional tool out of the two. REQUIREMENTS Fig 5: Requirements in ALM Fig 6: Sprint (usually two weeks) in a release for a project
- 7. 7 In the Image above I have numbered them from 1 to 8 to illustrate what they mean and how they are used.They have been described below: 1.Velocity at which Sprint is planned(units of worked planned to be completed in sprint) 2.Showing that this iteration of a particular sprint is has how many days remaining. 3.How much percentage of requirements/user stories have been delivered and accepted by the customer. 4.No of defects found in the Sprint. 5.Total active tasks still in the sprint.Tasks are defined within a user story upon completion of which a user story is marked as complete. 6.Burndown chart indicates how the team and work is progressing 7.All other activities like test cases planning ,test plan,defects,reports and various imp phases are present under these tabs. 8.Actual user stories/requirements written by business along with development on which the entire team works upon. DEFECTS Fig 7: Logging a Defect in ALM Fig 8 : Logging a Defect in Rally
- 8. 8 Similarly while logging defects in rally ,we need to keep in mind that all the relevant information is filled out as we used to when we fill it in ALM/QC.As noted above I have numbered them from 1 to 5 which are all described below: 1.Summary of defect 2.Description which will include detailed explanation and expected and actual output 3.All properties of the defect including attachment are entered here.Some of the fields will be mandatory as in ALM. 4.Team ,link to test case and assigned to will be covered here. 5.All comments by different stake holders are captured under discussion tab. Reports Generation Fig 9 : Reports in ALM Fig 10 : Reports in Rally All reports which were generated using ALM can also be generated using rally.There are some default reports present which are most commonly used in projects by scrum master and other team members.However we can always create our own reports by configuring the parameters based on our requirements. 3) How this paper will help understand the basic similarities and differences between the Waterfall and Agile model and will help to achieve quality work in future with a model suitable for their project?
- 9. 9 Fig 11: Cost and Schedule Driven vs Value Driven Fig 12: Value vs Risk with time Here are some of the main differences between Agile and Waterfall Models: The main advantage of the Agile model is the backward scalability. Under Waterfall approach it is not easy to change decisions which were made in the earlier stages. Any changes in the waterfall model demands the building of the entire project right from the start. The flexibility of error checking during any phase of development makes the Agile model the best approach for less erroneous results as compared to Waterfall model. The Waterfall model can only test bugs after the development of the modules. Agile model provides flexibility for making changes as per customer needs and guarantees client satisfaction. The Waterfall model suffers drastically in this respect since it does not allow any modifications midway through the development process. Agile development promotes modular partitioning with options for segregation. The rules are set down before the project is executed. The Agile approach supports simultaneous development of different modules as per client requirement.
- 10. 10 Fig 13: Waterfall vs Agile Here are some of the main similarities between Agile and Waterfall Models: Traditional and Agile Share the Same Goal. The Traditional World and the Agile World Use Many of the Same Principles institute multiple reviews move forward in a step-by-step, sequential manner move forward only when all parts of the previous steps were complete capture all details with extensive documentation The Traditional World and the Agile World Use the same basic building blocks scope cost schedule performance Once we are aware of the differences and similarities of the two models we can always chose the best that applies to the project. 4) Why it’s always a win –win situation with Agile compared to Waterfall model? In a perfect waterfall world, I will end up spending time determining exactly what the customer needs. Me and my team team will work hard developing/testing. Months or years would pass. Then, we would meet with our customer again and would reveal the most perfectly built system in the world. Our customer would say , “It’s exactly what I always wanted!” But remember how many times have we actually seen that happen? In the real world, this can go wrong if I have not fully understood my customer needs, documented vision does not fully encompass what is wanted, customer doesn’t have time to wait for the perfect end-result, my team runs into issues and the deadline keeps sliding, economic or business priorities shift while we are still in development/testing.
- 11. 11 Agile methodologies gave us the chance to analyse. One mis-conception with agile projects is that planning or fully developed requirements are not needed. In many ways, the requirements are better understood by the customer (because they often write them themselves) and by the development team (because they must agree the requirement is well written). The development team quickly agrees on how much effort each task will take, which can be much more accurate than a project manager’s estimates (depending on methodology) with a long release deadline. Another plus is that we have a better idea of the progress being made by the team. If our team uses the agile tool correctly, I’ll know within hours or days that a feature is at risk. Fig 14: User Experience changes from Waterfall to the Agile World Using an agile method, we understand as soon as our customers’ needs change and can shift our team’s efforts accordingly. This is because we are consistently communicating with them. Quicker release cycles mean our customer can see where features or functionality were missed and those can be re-prioritized over other less important items in the next cycle. The whole structure makes it easier to collaborate with our customer and makes them feel heard and in control. Other problems can arise in an agile project. It assumes we’ve hired good people and that they are all-in. It expects them to be responsible and develop with speed and quality in mind. Agile can require more discipline because it’s easier to lose focus on the big picture and instead drive for the delivery of features. Care must also be spent ensuring re- factoring and clean-up tasks are included in each release or every few releases. 5) Real Life Examples /Case Study/Images illustrating the two models and their challenges/benefits. If you see the below image closely enough, it will not take much of your time to understand how an agile model delivered a better product which complies with the user needs and expectation .This was possible because of intermediate reviews of the product with product owner and making appropriate changes as and when needed. This intermediate review was not possible with the waterfall model which resulted in delivering something which was not actually what the customer wanted.
- 12. 12 Fig 15: Waterfall vs Agile (Real life example of delivered product) CHALLENGES AND BENEFITS Fig 16 : Challenges and Benefits go hand in hand Challenges and Benefits go hand in hand when it comes to applying any methodology and benefiting out of it. But the benefits that we have from this new methodology outperform the challenges that we are facing. CHALLENGES: •Agile methodologies (e.g. Scrum, XP, Kanban, Crystal etc.) are often more difficult to understand than linear, sequential ones – at least initially. •Because of the emphasis on working software there can be a perception that documentation can sometimes be neglected. The focus should be on appropriate documentation to the audience that needs it but, if not implemented well, this isn’t always the case. •When implemented badly Agile can introduce extra inefficiencies in large organizations or can be working against long standing organizational processes. BENEFITS:
- 13. 13 •Working software is delivered much more quickly and successive iterations can be delivered frequently, at a consistent pace. •There is closer collaboration between developers and the business. •Changes to requirements can be incorporated at any point of the process – even late in development. •It gives the opportunity for continuous improvement for live systems •It is highly transparent CONCLUSION Today, after working for almost two years in agile software methodologies I am in huge favor of this methodology over others and I believe my experiences illustrated in this paper would make many of you agree to it. I also believe that technology, businesses and markets change so fast these days that software development needs to be adaptable above all other qualities and since Agile methods are more flexible than the waterfall method which means that customers’ requests are more likely to be met . Fig 17: Source: Version One State of Agile Survey 2013 Fig 18: Waterfall vs Agile (average success/challenged/failed) I also felt that the rhythm that Scrum creates helps to build highly motivated teams where productivity increases over time compared to traditional waterfall. Having said all that, there are still circumstances in which the waterfall method can be suitable – For example:
- 14. 14 Example 1: If an organization and the people involved in the project are not in a mature enough state for Agile it may be more appropriate to use traditional project management methods. Example 2: Where requirements are guaranteed to be unchanging and there is very little uncertainty or if the project if very simple – but those circumstances are becoming fewer and farther between. But these circumstances are becoming rarer with each passing year and believe me agile is here to stay and extend to most of the project. If only we do our homework well before moving to this methodology ,we will not only find the transition easy but also a more productive one. REFERENCE MATERIALS 1. https://en.wikipedia.org/wiki/Systems_development_life_cycle 2. http://info.110consulting.com/blog/bid/317786/Why-Move-from- Waterfall-to-Agile 3. http://manifesto.co.uk/agile-vs-waterfall-comparing-project- management-methodologies/ AUTHOR’S BIOGRAPHY Sourav Kumar is an enthusiastic Software testing professional and an active speaker on topics related to testing. He is an Engineering graduate and currently works as a Senior Engineer QE in UnitedHealth Group. He has around 7 years of experience in Specialized Software Data warehouse Testing and Automation. He is an ISTQB certified, ORACLE certified and an AHM certified Associate .His interests include automation, agile testing, ETL, Data warehouse, Healthcare and Banking Domain and other forms of testing. His hobbies include playing chess and table tennis. He has participated in TNS white paper event in 2012 , 2013 and 2014. He has also been awarded 5th (Topic: Testing on the Cloud) in TNS 2012 and 1st (Topic: Production Defect Reduction Strategies) in TNS 2013.
- 15. 15 These experiences have given him immense confidence to come up again and participate in TNS 2016.You can always reach out to him through his official mail id : sourav.kumar@optum.com if you are within UHG and mail id : sourav_nist@yahoo.com if you are from outside UHG. APPENDIX ETL – EXTRACT TRANSFORM LOAD RALLY-Rally is Software that provides agile software solutions that help streamline your agile business processes. IT - INFORMATION TECHNOLOGY SCRUM- Scrum is an agile software development model based on multiple small teams working in an intensive and interdependent manner SDLC-SOFTWARE DEVELOPMENT LIFE CYCLE