How to manoeuvre as test/QA responsible in agile teams to get the "right" product quality

AGILE DEVELOPMENT IN SAXO BANK
HOW TO MANEUVER AS TESTER
TO GET THE RIGHT QUALITY
BY KNUD HANGAARD

AGILE DEVELOPMENT IN SAXO BANK. HOW TO MANUVER AS TESTER TO GET THE RIGTH QUALITY

FACTS ABOUT ME
Master of Science, Electronics and
Computer Engineering
Certifications:
ISTQB, PRINCE2, ACI Dealing/Trading
My working sectors:
• HW/SW production industry
• Banking industry

2

My current role:
Front Office test lead:
• Release test responsible
• Test environments
• Development and test processes

Contact:
knud@hangaard.dk
LinkedIn:
www.linkedin.com/in/khangaard
SKYPE: khangaard


PRESENTATION FOCUS AREAS

Introduction to Saxo bank and the software
Agility challenges and finding the “Right” quality
What we have done to become more agile: process, tools, people
What to bring home:
o Key focus areas in the test process for achieving right level of quality
o Key skills that has been necessary to manage testing in a hyper agile organisation

3


INTRODUCTION TO SAXO BANK
Established 1992
Employees >1300 (700 in IT)
Offices 25 countries
FX trades daily 170,000
Daily av. turnover 20 billion USD
Countries with retail clients 190
Financial instruments > 25,000
Received prices daily 5-6 billion
Development in Denmark and India

The “Front Office” in Saxo Bank headquarter.

4


HUGE SOFTWARE PLATFORM

o Software components: 350+
o 11 million lines of code


THE DEVELOPMENT CONDITIONS
o FOCUS ON LEADING ROLE, DELIVERY AND COST:

o BE PRODUCTIVE
• Ongoing business development, large
project portfolio - demand for new
functionality (DEVELOPING THE BANK)
o BE GOOD
• Ongoing technical and performance
improvements – minimum number of
bugs in Live. (RUNNING THE BANK)

o BE ECONOMICAL
• Cost awareness: (off shore
development and test)

6


SO WE WANT TO BE AGILE…

7


SAXO BANK DEVELOPMENT MODEL
(Prince2)/Scrum/Kanban

Project
testers

Scrum/Kanban

Release
testers

Project 1

Projects

Project 2
Quality gate
Merge to main

Project N

Platform

Quality gate

Production

Defects

Incidents from Production
Project Test
Environments

8

Release Test
Environment

Simulation

Production


DEVELOPMENT TRENDS IN SAXO BANK,
BECOMING MORE AGILE
Delivery cycle has been reduced over time
o
Idea is less changes per cycle
o
No forcing of last minute changes (follow the process!)

Development has been divided into sectors, aligned with business areas
o
More focus on “the customer”

9


CHALLENGES FOR QUALITY ASSURANCE AND TEST
o
o
o
o
o
o
o

10

Huge code base
Huge amount of components
Test basis (documentation) is limited
Limited time for writing test cases
Limited time for testing
Limited time for writing test reports.
What would we really like to test - test coverage?


SO WHAT IS THE RIGHT QUALITY?
Live
Incidents
and roll
backs (y)

y=v*x where v: development velocity (throughput)

Low quality (many new
changes, huge amount of Live
incidents)
“Right” quality (enough number of new
changes, reasonable number of
incidents in Live)
High quality (less new
changes, only few Live
incidents)
Deliveries/changes to Business (x)

Right Quality:
o Happy Business (more competitive products, earnings – less problems in
Live, losses)
o Happy Customers (more products, opportunities – less bugs found)


BECOMING MORE AGILE
KEY POINTS COVERED:
o The Process
o The Tools
o The People

12


PROCESS

DEVELOPMENT AND TEST PROCESS – KEY POINTS
Time boxed development and time boxed testing (backlogs for each area)
Incremental approach:
o release in small steps - backward compatible - possible to roll back
Risk based testing:
o Spend more time on high risk changes (both functional and regression)
Flexible development and test process:
o Quality gates/ verification points can be decided on low level
Cross functional teams:
o Shared test responsibility between developers and testers

13

PROCESS


RELEASE TEST CYCLE (TIME BOXED)
o
o
o
o
o
o

Monday: Baseline
Tuesday: Deploy DB changes and verify
Wednesday: Deploy new services and verify
6 days for test and reporting
Next Thursday: Go/No meeting
Friday: Follow-up and planning next release
Monday

Tuesday

Wednesday

•Baseline of
test
environment

•DB release
candidates

•Apps og
Services rel.
candidates

Monday
•Release
Test

Tuesday
•Release
Test

Wednesday
•Release
Test
•Reporting

Thursday

Friday
•Release
Test

•Release
Test

Thursday
•Go/No Go
meeting
•Release to
Simulation

Friday
•Planning
next
release
test

Saturday

Sunday

•DB
Releases
to Live

Saturday
•Release
to Live

Sunday
Backup of
Live

PROCESS


RELEASE DOCUMENTATION BUILT-UP
A tree structure of work items:
o Release item: Parent for the full release

Release, week x
Change (Component 1)
Delivery (Project A)
User Story a

o Change items: One for each Component –
release request for production
o Delivery items: One from each team or project.

User Story b
Bug Fix y
Delivery (Project A)
User Story c

o User Story items: The functional changes

Change (Component 2)
o Bug items: The bug fixes.

Delivery (Project C)

…

15


STATE TRANSITIONS AND TEST FLOWS
Development User Story states (Kanban):
• Not Done, Active, Resolved and Closed
Independent Test stages (Test Flows):
• NA, Not Done, Done
Test flows at 3 stages:
• development phase,
• integration phase,
• system integration phase (release test)
Auto generated test report

16

PROCESS

TOOLS


THE TOOLS – ALL IN ONE (Team Foundation Server along
with Microsoft Test Manager)

Bugs

Test
Execution
Plan

Test
Results

17

Test Cases/
Test
Scripts

Test
Reports

Code

User
Stories/
Features

Tasks


TOOLS

THE TEST ENVIRONMENTS
o Test environments that simulates Live (Production) environment
• Release and project test environments
o Baseline of test environments with Live data every two weeks
o Intensive scripting for adapting Live data to test environment (object)
o One-click deployment of new software

18


PEOPLE

AGILITY - IMPACTS ON QUALITY ASSURANCE THINKING
o Adapt to the situation and plan very much based on RISK
o Find the “right” test depth (effort) – use your time optimal

o Share results early and adjust when you know more
• early feedback both ways is important
o Find the right level of test documenting and reporting
• so you get the message out but do not overdo

19


PEOPLE

A NEW TESTER ROLE
o Test certifications help as a basis - but is not sufficient in “the field”
o Be able to handle many different stakeholders (developers, project
managers, delivery managers, business owners, business support)
o Be technical well founded on multiple areas (less time for study, understand
fast, react fast)
o Be RISK focused (find the serious bugs in the beginning)
o Be Proactive - ask questions!
o Make plans but expect them to change and look for the new test opportunities
o Spend more time on exploratory testing instead of just running the same test
cases
o Do early and precise test reporting (only few lines to explain the problem)

20


EFFECTS OF THE APPROACH

21


EFFECTS
Business Effects:
• More releases going to Live
• Less high severity bugs found in Live (roll back)

Process Effects:
• Better transparency and traceability
• Effective documenting and reporting
• The flexible dev. model makes it easy to adapt to
new situations
• Benefit from time difference: Build late in
Denmark - test early in India
Test Effects:
• ”Exploratory testing” generates more bugs than
executing test cases prepared in advance

22


LESSONS LEARNED

23


KEY FOCUS AREAS IN THE TEST PROCESS FOR ACHIEVING
RIGHT LEVEL OF QUALITY
Simple, clear and flexible process
o Clear responsibilities
o Clear expectations
Teamwork
o Daily scrums with testers and
developers
o Avoid being dependent on other
team’s priorities.
Effective and clear communication
o To avoid time wasted
o To avoid bugs being overseen

24

Effective documentation
o Find the ”right” level of details
o Be focused on key points – make it
easy for the reader
Effective tooling
o Easy planning, test execution and
reporting
o Easy setup of test environments


KEY SKILLS NECESSARY TO MANAGE TESTING
IN A HYPER AGILE ORG.
o Great communication skills - be able to communicate with many different
stakeholders
o Knowledgeable
o Need to have both good business and technical/development knowledge
o Need to know the ”Go to Guys” to get more knowledge quickly
o Great team player - share knowledge – get knowledge
o Open mindset - be ready to change plans quickly
o Tester’s mindset – explore, ask questions
o Responsible mindset – Be the one that follows-up
25


CULTURAL DIFFERENCES BETWEEN OFF-SHORE TEAMS
Different thinking:
Communication path - hierarchical vs. lateral
Control - centralized vs. decentralized
Planning - detailed vs. less detailed
Different communication form:
Direct vs. indirect (verbally)
Direct vs. indirect (in writing – sugar coating)
language barriers
Different job goals:
Grow hierarchical vs. grow expert wise

26


CHALLENGES FOUND AND HOW TO OVERCOME
o Culture differences
o Discuss the cultural differences for better understanding and learning
o Engage with team members (non-work related)
o Travel often both ways
o Unclear goals/expectations
o Ensure clear communication (combine verbal and writing)
o Team SPOCs both places for ”translating” the messages
o Be very concrete on goals and expectations
o Regularly status meetings retrospectives and feedback
o Having the feeling of being far away
o Electronic boards for status sharing
o Video conferencing a must

27


NEXT PLANS FOR SAXO BANK
o Continuous delivery
o Even more effective processes
o Even more automation
o Even more auto generated reports

28


CONCLUSION – IMPORTANT FOCUS AREAS IN AGILE TEAMS
Clear Communication
Effective Processes

Effective Tools
Team Player Mindset
Risk Mindset


THANK YOU FOR LISTENING!

Feel free to contact me:
LINKEDIN: WWW.LINKEDIN.COM/IN/KHANGAARD
30

How to manoeuvre as test/QA responsible in agile teams to get the "right" product quality

More Related Content

What's hot (8)

Viewers also liked (20)

Similar to How to manoeuvre as test/QA responsible in agile teams to get the "right" product quality (20)

More from SQALab (20)

Recently uploaded (20)

How to manoeuvre as test/QA responsible in agile teams to get the "right" product quality