269 rosenblum -agile-it-org--ssp2008

Agile Development:
The Importance of Testing
Presented by
Bruce D. Rosenblum
CEO
Inera Incorporated

Society for Scholarly Publishers
30 May 2008

Real World Software Development

Credit: http://www.eos-india.net/abhisek/blog/softwaredevcycle-2006-06-14-20-38.html
© 2008, Inera Inc. All Rights Reserved

Software Development Goal
 Develop Applications
• Deliver value to the user
• Easy to use
• Provide reliable process


The Old Days: Waterfall Development

Scope Change
Feature Creep

Credit: http://www.csse.monash.edu.au/~jonmc/CSE2305/Topics/07.13.SWEng1/html/text.html

Why Waterfall No Longer Works
 Computers are more powerful
 Therefore projects are bigger

• And take longer to develop
 No longer possible to anticipate all requirements
• And cannot anticipate all changes during project
 By delivery time
• Customer interests changed
• Requirements changed
 “What I asked for, but not what I want”


Web Flexible World
 Old distribution model
• Disk shipments
• Waterfall development
• Slow, late deliveries
 New distribution model
• Internet delivery
• Automatic updates of client applications
• Continual update of ASP applications


New Development Methods
 Customer-centric

 Incremental changes
 Rapid Deliveries
 Continuous Testing


Agile Model


Agile Goals
 “Agile” means responsive to customer needs
 The goal is the customer, not the “project”
 Simplicity and simple steps


Getting Started
 Design simple task that does “something useful”
• Even if that task won’t be delivered to “customer”
 Start testing immediately
 Test early and often


Team Test Types
 Unit tests
• Test assumptions; vital for maintenance
 Integration tests
• Does it all work together correctly?
 Code Reviews
• Developers can learn from each other


Developers Can’t Test
 Developers are lousy testers
 Developers lie
• Is it done yet?
• When will it be done?
 Great developers test 2/3 of their time
• Write a little, test a lot


Customer Test Types
 Formal Quality Assurance
• Scripted test plans
 Ad hoc testing
• Sometimes called “monkey testing”
 Usability testing
• One-way mirrors
 User testing
• The importance of watching users
 “Run it yourself” testing


Test Administration
 Bug database
• TestTrack, Bugzilla, etc.
• FileMaker, Excel
 Short Cycles
• Daily bug review
• Fix bugs immediately
− New features must wait


Automated Testing
 Use of software to control testing
 Automatically run application features
 Results compared to previous “good”
result


Use of Automated Testing
 Inera relies heavily on automated testing
 Three months of development over 6 years
 Today: 30+ hours of automated testing per night
 Manual validation performed on all releases


Example: CrossRef Testing
 Inera
implemented CrossRef query module
 Automated testing of CrossRef module

• Queried ~1000 references per night
• Compared DOIs with previous “good” results
 Sometimes links go away
• Usually incorrect deposit (e.g. metadata conflict)
 Sometimes new links appear
• Replaces previous “good” results


Impact of Automated Testing
 New problems caught early
• Problems found quickly when code is changed
 Stability of code maintained
 Fixes are never “lost”
 Short release cycles


Cost of Automated Testing
 Sustainable and scalable model
• Up-front investment yields large dividends
 Lower costs
• QA cost ~ $60,000/person/year
• Versus 30+ hours of PC time per night
− 4 PCs + electricity + 3 months engineering time

 No fulltime QA personnel
• Manual testing performed by multiple team members


Time for Testing
 Testing does NOT take extra time
• Even manual testing
 Testing reallocates time
• Less time spent tracking latent bugs
• Less time reworking code to fix bugs
 Great testing allows faster releases


“True” Cost of Automated Testing
 Faster Release cycles
• Can release latest code daily if stable
 Fewer support calls
 Happier customers


Who Tests?
 Developers

 Quality Assurance Specialists
 Customer

 Actual Users
 Machines (automated testing)
 So who tests? The Agile Team


Conclusions
 System stability requires constant testing
 Great testing lowers costs
 Great testing delivers software faster


Questions?

Bruce Rosenblum
Inera Incorporated
+1 (617) 969 - 3053
brosenblum@inera.com
www.inera.com


269 rosenblum -agile-it-org--ssp2008

More Related Content

What's hot (20)

Viewers also liked (6)

Similar to 269 rosenblum -agile-it-org--ssp2008 (20)

More from Society for Scholarly Publishing (20)

269 rosenblum -agile-it-org--ssp2008