STQA2 presentation about STQA by given university

STUDY IN ENGLISH
STUDY IN ENGLISH
www.study.singidunum.ac.rs
Integration testing
Miodrag Zivkovic

STUDY IN ENGLISH
Integration testing
• Unit vs integration testing?
• Unit testing is based on testing individual units of
source code
• Unit – the smallest part of the application which
can be tested
• In OOP, unit is usually one method or one class
• Unit testing is usually performed by developers

STUDY IN ENGLISH
Integration testing
• Integration testing is testing phase where individual
modules are connected and tested as a group
• It is performed after unit testing, but before system
testing
• Since integration testing is performed on subset of
all modules of the complete project, it is not the
same as system test

STUDY IN ENGLISH
Integration testing
• Integration testing starts with the components
which passed unit testing
• They are combined in larger groups
• Tests defined in integration testing plan are applied
• As result, at the end of the integration testing we
have integrated system ready for system testing

STUDY IN ENGLISH
Integration testing
• Why integration testing is important?
• After all, all units and modules are already tested in
details on unit level
• Inevitably there will be new problems
• Several components now working together for the first
time
• If integration is performed badly, all problems will
arise at once, which is hard for testing, debug,
fixing etc.

STUDY IN ENGLISH
Integration testing
• Biggest problem in integration testing is connection
of the components
• Data can be lost while transferred through interface
• One component can have bad infulence on another
• Some combination of sub-functions can give wrong
result of the main function
• Inaccuracy of components acceptable on unit level, can
be multiplied and unacceptable on integration level
• Global data structures can be an issue

STUDY IN ENGLISH
Example
• September 1999, Mars Climate Orbiter mission
failed after successful travel of 416 million miles for
41 weeks.
• Craft was damaged and lost communication while
entering the orbit around Mars
• Defect should have been detected by integration
testing. Lockheed Martin Astronautics used
imperial units (pounds) for calculating thrust, and
Jet Propulsion Laboratory used metric units for
calculations (Newtons)

STUDY IN ENGLISH
Integration testing
• We want to ensure that our components meet
requirements:
• Functional
• Performance
• Robustness

STUDY IN ENGLISH
Integration testing
• Big bang integration (Run it and see approach)
• Individual modules are developed and tested
separately (on unit level)
• Integration starts only when all components are
finished and ready
• Then, they are all connected and integrated at once

STUDY IN ENGLISH
Integration testing
• Usually results in a complete chaos

STUDY IN ENGLISH
Integration testing
• Problems on the interfaces of the components are
detected very late
• It is hard to isolate defect, and to determine
whether defect is in component or in the interface
• There is a big chance that some critical bugs will
not be found, and they will usually be found by
customer in production
• It is impossible to determine if all necessary tests
for integration are covered

STUDY IN ENGLISH
Integration testing
• Big bang – not a good way to integrate and test
software
• Integration is performed without any formal
integration testing, and executed blindly with best
hope that all components will work perfectly in
combination with others
• It is applicable only for very small systems (few
components), not for more serious applications

STUDY IN ENGLISH
Integration testing
• Incremental integration
• Functional skeleton of the system is developed first
• Every new module is designed, developed and tested
• Afterwards, this module is integrated with the skeleton, and
tested thoroughly before adding any other new module

STUDY IN ENGLISH
Integration testing

STUDY IN ENGLISH
Integration testing
• Advantages of incremental integration – easy finding,
isolating and fixing errors
• It is appropriate to the incremental model of software
development – system is always in (relatively)
operational state, and can be shown to customer
• Disadvantage – sometimes it is needed to create stubs
and drivers, which will go in place of components which
are not yet developed or integrated

STUDY IN ENGLISH
Integration testing
• What are drivers and stubs?
• We observe system architecture and identify
hierarchy and layers

STUDY IN ENGLISH
Integration testing
• Some components in the hierarchy can be incomplete
or not implemented
• Instead of working with real components, component
under test communicate with simulated components
which either call it, or respond to calls on the same way
as real components
• Based on the fact if the components are calling other
components, or are being called by other components,
they are divided into two types

STUDY IN ENGLISH
Integration testing
• Driver – component which simulates module of the
higher level, which calls other components and
waits for some response
• Doubler (stub) – dummy module which simulates
module of the lower level. It simulates real
component which is called and return same result
as the real component

STUDY IN ENGLISH
Integration testing
• Component under test is taken from the system context in
which it communicates with other components, and placed
in the test context (unit testing in general)
• Component T under test, in real environment, is called from
components A and B, and it calls components C and D

STUDY IN ENGLISH
Integration testing
• Test doublers can be of different types and complexity:
Dummy – Empty doubler, simplest type. Enables connecting of
the program, and provides default (constant) value when called.
Stub – more complex than dummy, has simplified logic added,
and provides different values when called.
Spy – spies parameters and internal state of the component,
and provides advanced validation.
Mock – fake component, which validates specific behavior,
mostly verifying values of parameters in the call, and order of
calling the component.
Simulator – realistic approximation of the component, usually
requires a lot of effort to be developed.

STUDY IN ENGLISH
Integration testing
• How to implement drivers and doublers?
• Usually programmed, except in case that driver is
actually human tester manually calling components
of graphical interface
• For implementation of driver, we can use JUnit (for
Java), NUnit (for C#) etc.

STUDY IN ENGLISH
Integration testing
• Doublers are usually implemented by defining
components which will simulate real components,
and define inside them what to return for certain
sequences of input data
• There are various libraries which support creating
such components, such as JMock, Mockito …

STUDY IN ENGLISH
Integration testing
• Both drivers and doublers are additional cost
• It is also software which has to be developed (by
usual methods), but not delivered with the final
version of program
• If drivers and doublers are simple, additional cost is
small

STUDY IN ENGLISH
Integration testing - types
• Types of integration testing based on the
hierarchical structure of the program:
• Top – down integration
• Bottom – up integration
• Mixed (sandwich) integration

STUDY IN ENGLISH
Top – down integration
• While integrating components, we move downward
the control hierarchy
• We start with the main control module

STUDY IN ENGLISH
• Incremental technique, after testing the main module
we add lower modules one by one
• Lower levels are generally simulated by stubs
• When new module is added, stub is replaced with real
component
• Modules below the main module can be added by one
of two possible strategies:
• Depth first
• Width first

STUDY IN ENGLISH

STUDY IN ENGLISH
• Strategy depth first will integrate all components on the
main path of the structure. Choice of path depends of the
program characteristics
• Strategy width first includes all components directly below
in one level, moving through the structure horizontally

STUDY IN ENGLISH
• Integration is performed in the following way
1. Main modul is used as test driver, all components below the
control module are replaced with stubs
2. Depending of the strategy, stubs are replaced with real
components one by one based on the order defined by strategy
3. Tests are performed after every integrated component
4. After successful testing, stub is replaced with real component
5. We can perform regression testing to make sure no new errors
have been introduced
• We repeat the process from step 2 until we cover complete
structure of the program

STUDY IN ENGLISH
• Depth first:
• Choosing left path, first
components to be integrated
are: M1, M2, M5
• Then M8, or (for proper
operation of M2) M6
• Then we implement and
integrate middle and right
path

STUDY IN ENGLISH
Top - down integration
• Width first:
• First integrate M2,
M3 and M4
• Next control level
M5, M6, ...

STUDY IN ENGLISH
• Advantages:
• Early start of testing and integration of components
• Reduced cost of driver development
• Components can be developed in parallel
• Disadvantages:
• Large number of stubs
• Stubs must be well written (testing depends on them)

STUDY IN ENGLISH
Bottom up integration
• Implementation and testing starts from the lowest
modules in hierarchy
• We move upward, toward main module, which is
integrated last

STUDY IN ENGLISH
• Since integration is done from the bottom, data
computing performed on the lowest levels will be
available early
• No need for stubs
• However, we need drivers, which will control and
send relevant data to lower modules

STUDY IN ENGLISH
• Implementation is usually consisting of the
following:
1.To complete some subfunction, lowest level
components are connected to clusters
2.To coordinate a cluster we write a driver
3.We test the cluster
4.When higher level component is ready, we replace
driver with real module

STUDY IN ENGLISH
• We group components
into clusters 1, 2 and 3.
Each cluster is tested
separatedly, by using
drivers
• Components in clusters
1 and 2 are below
module Ma.
• After testing drivers D1
and D2 are removed,
and clusters 1 and 2 are
connected directly to
Ma when it is ready

STUDY IN ENGLISH

STUDY IN ENGLISH
• Disadvantages:
• The biggest cost is driver development
• Driver does not test interface to components directly
• Verification of critical control structures is postponed to
the end of development cycle

STUDY IN ENGLISH
• Advantages:
• Testing starts early
• As soon as the first component – leaf is ready, we can
start testing
• It is possible to write code and test in parallel
• Need for stubs is reduced

STUDY IN ENGLISH
Sandwich integration
• Combines both approaches: bottom up and top - down
• System is observed as it “has” 3 layers
• Target layer in the middle
• Layer above => top down approach
• Layer below => bottom up approach
• Testing converges to the target layer
• If there is more than 3 layers, how to select target
layer?
• Try to minimize number of stubs and drivers

STUDY IN ENGLISH
A
B C D
G
F
E
Layer I
Layer II
Layer III

STUDY IN ENGLISH
• Advantages:
• Testing of upper and lower layer can be done in parallel
• Stubs and drivers are not needed for upper and lower
layer, so total number of stubs and drivers is reduced
• Integration of components is possible imediatelly after
implementation

STUDY IN ENGLISH
• Disadvantages:
• We still need some „throw-away“ code
• Approach looks a bit like big bang testing, within sub
trees
• Harder defect isolation
• Solution – modified sandwich strategy

STUDY IN ENGLISH
Modified sandwich integration
• All three layers should be tested individually
• After that, given layers are integrated
• Test for each layer are the following:
• Target layer is tested with drivers and stubs
• Upper layer is tested with stubs
• Lower layer is tested with drivers
• We integrate the layers in the following way:
• Test interaction of upper layer with target layer
• Upper layer replaces drivers
• Test interaction of lower layer with target layer
• Lower layer replaces stubs

STUDY IN ENGLISH
Example
• We observe calendar application system
• Format mm, dd, yyyy
• Functionalities
• Next Date
• Day of the week corresponding to the given date
• Latest Friday the 13th
• ….

STUDY IN ENGLISH
Example
• Functional decomposition of the problem

STUDY IN ENGLISH
Example
• Top-down integration, first step (stubs are gray). On this
level we check main program

STUDY IN ENGLISH
Example
• Stub representing weekDay

STUDY IN ENGLISH
Example
• Next 3 steps (top-down integration, width first)

STUDY IN ENGLISH
Example
• Bottom-up integration – starts with leaves in functional
decomposition tree

STUDY IN ENGLISH
Example
• Bottom-up integration of module which calculates zodiac
sign. Calling component (in this case main) is replaced with
driver.

STUDY IN ENGLISH
Example
• sandwich integration

STUDY IN ENGLISH
Example
• Big bang – everything at once

STUDY IN ENGLISH
System testing

STUDY IN ENGLISH
System testing
• System testing is based on testing of the complete
system as a whole, after complete integration
• It is defined by functional requirements and system
specification
• Additionally includes test based on risk, business
processes, usage scenarios, use cases or other high
level system descriptions, interactions with other
systems and system resources

STUDY IN ENGLISH
System testing
• Usually final test to verify that system as a whole meets
specification requirements and its purpose
• Done by testers specialists, or even independent testing
teams
• Tests are performed from the end to end perspective
• Since it is based on system specification, it is black box
method
• Both functional and non functional requirements are
tested

STUDY IN ENGLISH
System testing
• After system testing, most of the remaining defects
should be detected and fixed, and system is ready
for client delivery
• Client usually initiates Acceptance testing, or User
Acceptance Testing (UAT)

STUDY IN ENGLISH
UAT
• Acceptance testing (UAT) usually is performed by
clients
• Goal of UAT is to establish trust into the system
• Focus is on checking functionality, by validating if
the system is adequate for the end user

STUDY IN ENGLISH
Alpha testing
• Part of UAT
• One of most commonly used software testing
strategies
• Testing is performed on the site of the company
developing software, where programmers observe
simulated users and note problems
• It is done when application development is near
end. It is still possible to do minimal changes to the
design of application as the result of alpha testing

STUDY IN ENGLISH
Alpha testing
• Alpha testing is typically performed by the group of
independent testers (not involved in previous
testing phases) but within the same company
• Last level of testing before delivering application to
the client
• Goal is to simulate real users as much as possible,
by using black and white box techniques, and to
find eventual bugs and fix them before delivery

STUDY IN ENGLISH
Beta testing
• Another name Field Testing
• Performed on the client site
• Software is delivered to the limited number of real
users, who install it and use it in real conditions
• Goal is to find any bug from user perspective, which
we don’t want to have in final application
• Most of the companies, such as Microsoft for
example, give beta version to users for testing

STUDY IN ENGLISH
Beta testing
• Open and closed beta

STUDY IN ENGLISH
Beta testing
• Advantages:
• Opportunity to give application to real users before releasing it to
general public
• Users during beta can test application and send feedback
• Beta testers often find errors not detected until then (from the user
perspective), ant it is possible to fix them before global release
• When those errors are fixed, software quality is better after going
to the market
• It also positively affect customer satisfaction

STUDY IN ENGLISH
STUDY IN ENGLISH
Software development
models
Miodrag Zivkovic

STUDY IN ENGLISH
Software development models
• Different processes and methodologies, which are
chosen and used in software development,
depending of the requirements and goals of the
project
• There are number of models developed in the past
years, each with different goals
• Generally, models specify different phases in
development, and the order of their execution

STUDY IN ENGLISH
• Choice of the model to be used in development of
the certain software has large influence on the
testing
• Chosen model defines what, where and how it will
be tested, affects regression testing, and choice of
techniques which will be used
• Some of the most frequently used models are
shown on the next slide

STUDY IN ENGLISH
• Waterfall model
• V model
• Incremental model
• RAD model
• Agile model
• Iterative model
• Spiral model
• Prototype model

STUDY IN ENGLISH
• Also called Software Development Process Models
• Every model follows software life cycle to ensure
successful finish of the project
• All models describe phases in software development,
and their order
• Software testing team follows Software Testing Life
Cycle (STLC, similar to software development model)
• Each phase has certain result (deliverables), which are
entry to the next phase

STUDY IN ENGLISH
• What are deliverables?
• Example:
• Requirements are translated to design
• Code is written based on the design
• After coding, testing verifies implementation against the
requirements

STUDY IN ENGLISH
Phases
• Generally speaking, there are 6 phases in every
software development model
1.Requirements gathering and analysis
2.Design
3.Implementation (coding)
4.Testing
5.Delivery (deployment)
6.Maintenance

STUDY IN ENGLISH
Requirement gathering and analysis
• Client requirements are gathered in this phase
• The most important phase for project managers
and stakeholders
• Stakeholder – someone who invested in business,
and has interest that this business would succeed
• Meetings are held between managers, stakeholders
and clients
• Goal is to gather basic requirements

STUDY IN ENGLISH
• Basic requirements are typically:
• Who will use system?
• How it will be used?
• What are input data?
• What are output data?
• This phase should answer those basic questions

STUDY IN ENGLISH
• After gathering requirements, they are analyzed
and it is considered hot to incorporate them in the
system to be developed
• At the end, formal document, called requirement
specification is created, which is input for the next
phase of the model
• Software testing team which follows STLC, starts
with the testing planning phase as soon as
requirement analysis is finished

STUDY IN ENGLISH
Design
• In this phase design of the system and software is
prepared based on the specification from the first
phase
• Hardware and system requirements are specified,
and general architecture of the system is defined
• This specification is entry to the next phase of the
model
• In this phase, testers define testing strategy – what
to test and how to test

STUDY IN ENGLISH
Implementation
• After getting specification of the system design,
work is separated into modules (units) and
programming starts
• Code is produced in this phase, and it is the most
important phase for developers
• This is the longest phase in software development
model

STUDY IN ENGLISH
Testing
• After the code has been developed, it is tested
against the requirements to ensure that the
software is really meeting client needs defined in
requirements gathering phase
• In this phase all types of functional, unit,
integration, system and acceptance testing are
performed
• It also includes non functional testing

STUDY IN ENGLISH
Deployment
• After successful testing, product is delivered to
client
• After first delivery, client will usually do beta testing
• If there is a need for additional change, or a bug is
found, it will be reported to dev team
• When all changes are implemented and all bugs are
fixed, final deployment is performed

STUDY IN ENGLISH
Maintenance
• When client starts to use developed system, often
some problems arise which should be fixed from
time to time
• This process of maintaining developed system is
called maintenance phase

STUDY IN ENGLISH
• Choice of appropriate model for development of
concrete application is critical
• Processes of development ant testing are based on
model
• Different companies choose different models,
which are appropriate for the applications they are
developing
• Today, on the market, most commonly used
methodology is Agile model

STUDY IN ENGLISH
• Waterfall model – very old model, where testing
starts after development is finished
• Since a lot of bugs are reported at the end of
development, cost of their fixing is usually high, so
people are now preferring Agile model (after every
sprint – demo of the feature to the client)
• Large number of companies still use V-model
• Incremental models are also used frequently

STUDY IN ENGLISH
Waterfall model
• First model introduced
• Also called linear sequential model
• Easy to understand and to use
• Each phase must be completed before starting of the next
phase (no phase overlapping)
• It can be used for small projects with clear requirements
• At the end of each phase, formal revision takes place to
ensure if the project is on right path
• Testing starts only when development is completed

STUDY IN ENGLISH
Waterfall model

STUDY IN ENGLISH
Waterfall model
• Advantages:
• Easy to understand and use
• Easy for management because of model rigidity,
every phase has specific results and revision
• Phases are processed and completed one by one,
no overlapping
• Very useful for smaller projects with clear
requirements

STUDY IN ENGLISH
Waterfall model
• Disadvantages:
• When application is in testing phase, it is hard to go
back and change something which has not been
thought through in earlier phases
• Operational sw is achieved late in the model
• High risk
• Not good for complex and OO projects
• Bad for long projects
• Bad in case of frequent requirement changes

STUDY IN ENGLISH
Waterfall model
• When to use:
• Clear requirements, which are known and fixed (no
changes)
• Known technology
• Short project

STUDY IN ENGLISH
Waterfall model
• Very little interaction with the customer during
development
• Only when project is finished, it can be shown to
end users
• If serious errors arise at that point, cost of their fix
is very high

STUDY IN ENGLISH
V model
• Verification and validation
• Similar to waterfall, sequential processing of the
phases
• Every phase must be completed before starting
next one
• Testing is planned parallel to the corresponding
development phase in V model

STUDY IN ENGLISH
V model
• Requirements are analyzed at the beginning. Before
starting of the development, system testing plan is
created
• This plan is focusing on specified functionalities,
defined in requirements gathering phase

STUDY IN ENGLISH
V model
• High level design – HDL, focused on system
architecture and design
• Overview of solutions, platforms, systems, products
and services needed to be implemented
• Integration testing plan is created in parallel, to
ensure all parts are working together

STUDY IN ENGLISH
V model
• Low level design - LLD is focusing on the design of
sw components
• It defines logic for every component of the system,
and class diagram is created with all methods and
relations between classes
• Component test plan is created in parallel

STUDY IN ENGLISH
V model
• Implementation phase – complete development
• When development is finished, execution path goes
to the right side of V model, where previously
created test plans are put to use
• Coding is at the bottom of V model, where
component design is transformed to code
• Unit testing is performed by developers which are
writing the code

STUDY IN ENGLISH
V model
• Advantages:
• Easy and simple to use
• Some testing activities are done before coding,
such as planning and test design, which saves a lot
of time
• Possible to find defect early
• Good for small projects, with clear requirements

STUDY IN ENGLISH
V model
• Disadvantages:
• Very rigid and not flexible
• Software is developed in implementation phase, no
early prototype to show to the customer
• If requirement change happens mid way, all test
documents must be updated together with
requirements

STUDY IN ENGLISH
V model
• When to use:
• Small to medium projects, with clearly defined and
fixed requirements
• High level of trust from the client is needed,
because they will not have prototype early enough

STUDY IN ENGLISH
Incremental model
• Development is separated into builds
• Multiple cycles of development, basically this is multiple waterfall
model
• Cycles are divided into smaller modules which are handed more easily
• Each module goes through phases of requirements defining, design,
implementation and testing
• Work, operative version is produced in first module, so operative sw
exists early in the module
• Each release of the new module adds new functionality to the previous
release
• Process is repeated until complete system has been implemented

STUDY IN ENGLISH
Incremental model
• Example
• Parts are added incrementally, every part is completely
finished
• Parts are added until project is completed
• In first iteration, first module is completely finished and can
be shown to client
• In second iteration, second module is finished and
integrated with first …

STUDY IN ENGLISH
Incremental model

STUDY IN ENGLISH
Incremental model
• Advantages:
• Operative software exists very fast and early
• Flexible – change of requirements is costing less
• Easy for testing
• Client can participate and give feedback for each build
• Initial delivery to the client is costing less, because client has
already seen previous builds and knows what to expect
• Easy for risk management, because risky modules can be
identified and developed with special care

STUDY IN ENGLISH
Incremental model
• Disadvantages:
• Good planning and design are needed
• Clear and complete definition of the system is
needed before dividing it to parts
• Overall cost is bigger than in waterfall

STUDY IN ENGLISH
Incremental model
• When to use:
• Clear requirements for complete system are
available
• Main functionalities are defined, some details can
be developed over time
• It is required to release product early on the market
• New technology is used
• There are risky functionalities and/or risky goals

STUDY IN ENGLISH
Agile model
• Type of incremental model
• Software is developed in incremental, short cycles
• As a result we have small incremental releases,
every adds something to the previous functionality
• Every release is tested thoroughly to ensure
software quality
• Used for time – critical applications

STUDY IN ENGLISH
Agile model
• Advantages:
• Client satisfaction is ensured with fast, continual deliveries of working
software
• Accent is on the people and interactions, not on process and tools.
Clients, developers and testers constantly cooperate
• Working software is delivered frequently – weekly based
• Daily based cooperation between developers and business people
• Continual focus on good design and technical corectness
• Adaptability to changes
• Even late changes in requirements are welcome

STUDY IN ENGLISH
Agile model
• Disadvantages:
• In case of large sw components, it is difficult to estimate the
effort
• Focus is not on design and documentation
• Projects can easily go in wrong direction if the client is not
clear enough about the final goals
• Only senior developers are capable to make decisions
during development, so it is not a good place for young
programmers (except in case they are coupled with very
experienced resourses)

STUDY IN ENGLISH
Agile model
• When to use:
• Fast implementation of new functionalities
• Very little planning needed to start the project – we
expect that the needs of the end user will
constantly change
• Client has constant contact with the product and
influence on the development

STUDY IN ENGLISH
Iterative model
• We don’t start with full req specification
• We start with the specification and implementation
of the part of the software, and then analyze it to
identify new requirements
• Process is repeated over and over, and new
software version is produced for each cycle

STUDY IN ENGLISH
Iterative model
• First rough sketch is created, and then iteratively
analyzed and improved in following iterations

STUDY IN ENGLISH
Iterative model

STUDY IN ENGLISH
Iterative model
• Advantages:
• We can start from high level design
• Software is developed step by step
• It is possible to get feedback from the client fast, by
demoing sketches and templates
• Less time spent on documentation, more on design

STUDY IN ENGLISH
Iterative model
• Disadvantages:
• Rigid phases, no overlapping
• Expensive errors in architecture or design are
possible, because we don’t have all requirements
before starting of the development

STUDY IN ENGLISH
Iterative model
• When to use:
• Clear requirements for complete system
• Large project
• Main functionalities defined, some details can
evolve during time

STUDY IN ENGLISH
Spiral model
• Similar to incremental model, with larger focus on risk
analysis
• 4 phases:
• Planning, risk analysis, development and evaluation
• Software goes through those phases multiple times in
iterations (spirals)
• Spiral starts with planning phase, where requirements
are gathered and risk is analyzed
• Following spirals are concatenated to the first one

STUDY IN ENGLISH
Spiral model
• Phases:
• Planning – requirements are gathered
• Risk analysis – risks are identified, together with alternative
solutions. At the end of this phase prototype is produced. If
risk is found, alternative solutions are suggested and
implemented
• Development – software is developed in this phase,
together with testing at the end of the phase
• Evaluation – In this phase client evaluates project, before
continuing with the new spiral

STUDY IN ENGLISH
Spiral model
• Advantages:
• Frequent risk analysis, therefore high probability to
avoid it
• Good for big and critical projects
• Strong control of documentation and client validation
process
• Additional functionality can be added in later spirals
• Software is produced early

STUDY IN ENGLISH
Spiral model
• Disadvantages:
• Can be very expensive
• Adequate risk analysis it demands high and
expensive experience
• Project success depends of the risk analysis phase
• Not good for small projects

STUDY IN ENGLISH
Spiral model
• When to use:
• All costs and risk evaluation are important
• Good for medium to high risk projects
• Clients are not sure in their needs
• Very complex requirements
• Large changes are expected

STUDY IN ENGLISH
•Agile methodology

STUDY IN ENGLISH
Agile methodology
• Agile methodology is sw development process
(such as other models, Waterfall, V, iterative…)
• However, it is significantly different from others
• Agile – English for ability to move easily and lightly,
and to easily answer to any change
• That is the key aspect of the agile development

STUDY IN ENGLISH
Agile methodology
• In case of traditional models, such as Waterfall, it is
usually required several months or even years to
complete the project, and the client usually don’t
see the product until the end
• Significant time is spent on requirements gathering,
design, development, testing and User acceptance,
before delivery

STUDY IN ENGLISH
Agile methodology
• Opposite to that, agile projects have sprints
(iterations) which are short (usually between 1-2
weeks and one month)
• All defined functionalities are developed and
delivered within the sprint
• There can be more iterations, and delivery of the
complete project is done after final iteration

STUDY IN ENGLISH
Agile methodology
• Example – Google works on the project of
developing product concurrent to MS Word, which
will have all functionalities of MS Word and any
other functionality defined by marketing team
• Final product must be ready in 10 months
• Traditional vs Agile?

STUDY IN ENGLISH
Agile methodology
• Traditional Waterfall model
• 15% time spent on requirements gathering and analysis
(1.5 months)
• 20% time spent on design (2 months)
• 40% coding and unit testing (4 months)
• 20% system and integration testing (2 months)
• At the end, 2 weeks spent on User acceptance from
marketing team
• Client would not be able to see the end product until the
very end of project, when it is to late to make changes

STUDY IN ENGLISH
Agile methodology

STUDY IN ENGLISH
Agile methodology
• Agile:
• Project is divided to sprints
• All sprints are same length (2-8 weeks for example)
• After every iteration functional product is released
• In simple scenario, project would be divided into 10
sprints, with 10 releases (sprint 4 weeks)

STUDY IN ENGLISH
Agile methodology
• Agile:
• Instead of spending 1.5 months on requirements
gathering, team decides on basic functionalities needed
for the product and develops them in first iteration
• Other functionalities which cannot be delivered in first
iteration, are planned for the next iterations based on
their priority

STUDY IN ENGLISH
Agile methodology
• Agile:
• At the end of the first iteration, team delivers operative
software with functionalities implemented in first sprint
• There will be 10 sprints, after every one working
software is delivered to client, incrementally enhanced
with new functionalities

STUDY IN ENGLISH
Agile methodology
• This approach allows client to interact and work
with functional software at the end of each sprint,
with possibility to give feedback
• Team can adapt to changes easily and do
corrections if needed
• Software is developed and delivered incrementally
through sprints

STUDY IN ENGLISH
Agile methodology
• In agile, more accent is given to the collaboration
inside team and cooperation with the customer, to
be able to easily answer to changes
• Agile development is currently most popular
methodology in IT
• Statistics show that over 50% of companies use
some sort of agile development

STUDY IN ENGLISH
Agile methodology
• In traditional development, every phase must be
completed before going to next one
• For example, when requirements gathering is
finished, we move to design phase, then after
completing that phase we go to development and
to testing at the end
• No overlapping of the phases, errors found late can
be very expensive

STUDY IN ENGLISH
Agile methodology
• In case of agile, every functionality is completed in
terms of design, development, coding, testing and
fixing, before it can be announced completed
• No separate phases, everything is done inside one
phase only

STUDY IN ENGLISH
Agile methodology
• Advantages:
• Continuous delivery
• Customer satisfaction, after every sprint there is a functional sw
• Client can give feedback or to request requirement change which
can be included in the next delivery
• Focus on interaction between clients, programmers and testers
• Focus on good design
• Any change in requirements is welcome, even in late stages of
development

STUDY IN ENGLISH
Agile methodology
• Disadvantages:
• Often poor documentation
• Requirements can be unclear, so it is hard to anticipate
expected result
• Difficult to estimate effort for implementation of a function
• Project can easily go in wrong direction
• Only senior developers are capable to make decisions
during development

STUDY IN ENGLISH
Agile methodology
• In agile team testers are observed as members of the
team, they are not identified based on specialization or
skills
• Both testers and programmers together are called
Development team
• Development team therefore is consisting of
programmers and testers who are actively involved in
product development
• Members of the agile team are communicating often in
order to achieve high quality product

STUDY IN ENGLISH
Agile methodology
• Agile Manifesto – 4 aspects
1.Individuals and interactions over processes and tools
2.Working software over comprehensive documentation
3.Customer collaboration over contract negotiation
4.Responding to change over following a plan
• Bold items on the left have larger value than items on
the right side

STUDY IN ENGLISH
Agile methodology
• Most important item is related to functional
software
• In traditional models, such as Waterfall, practice is
to finish design, system architecture documents,
test plans first, before delivering real piece of
working software

STUDY IN ENGLISH
Agile methodology
• In agile methodology, this practice is changed by
delivering working software increment in each
iteration
• It does not mean that documentation should be
neglected completely, but to create only necessary
documentation
• Software should be delivered early and as often as
possible to the client during the development

STUDY IN ENGLISH
Agile manifesto

STUDY IN ENGLISH
Agile methodology
• Common approaches?
• Several frameworks used in companies
• All of them based on agile manifesto
1.Scrum
2.Kanban

STUDY IN ENGLISH
Scrum
• Software development is divided into iterations -
sprints, usually fixed at 1-4 weeks
• Potentially deliverable (and tested) software
increment should be built in each iteration
• Sprint length is decided based on the team,
requirements and capacity
• Once decided, sprint length should not be changed

STUDY IN ENGLISH
Scrum
• At the end of each sprint team delivers potentially
shippable software, which is tested and working
• Since sprints are short, only important
functionalities should be developed at the
beginning
• Client can see basic functionalities fast and give
feedback

STUDY IN ENGLISH
Scrum
• Product backlog – set of all requirements is divided
to smaller work units (tasks) and prioritized into the
list
• Sprint backlog contains set of all requirements
which are in the current sprint
• Task with highest priority is on the first place
• Team members take tasks and work on them
• This is actually list of all tasks which should be finished
inside sprint, sorted by priorities

STUDY IN ENGLISH
Scrum
• Daily Stand Up Meeting (Scrum Meeting)
• Teams hold daily meeting
• Every team member gives short update on 3 questions
to the rest of the team
• Questions – What I have finished yesterday, what I am
doing today, is something blocking me to continue?
• Meeting last 15 minutes, held every day at the same
time and same place

STUDY IN ENGLISH
STUDY IN ENGLISH
Regression testing
Miodrag Zivkovic

STUDY IN ENGLISH
Regression testing
• During iterative software development, in each
iteration we implement a part of the system
functionalities and we focus testing on those
functionalities
• Problem – functionalities which were implemented,
tested and verified in earlier iterations can stop
working properly
• Why? During current iteration some part of the
application can be changed and it can have negative
influence on previously implemented functionalities

STUDY IN ENGLISH
Regression testing
• It is logical (at first glance) to focus testing on the new
functionalities
• Functionalities implemented in the current iteration are
certainly the most important for testing
• However, if we blindly test just the new functionalities,
we will verify only what was implemented in the
current iteration
• All previously implemented functions can stop working
properly, and nobody would notice (because nobody
was checking them at all)

STUDY IN ENGLISH
Regression testing
• Regression testing is a testing method where we repeat
tests executed in earlier iterations to verify if the
functionalities implemented in earlier iterations are still
working properly
• In the first iteration we test just the functionalities
implemented in it
• Already in the second iteration we test not just
functionalities implemented in the current iteration,
but the functionalities tested in the first iteration as
well

STUDY IN ENGLISH
Regression testing
• In general, in each iteration we test not just the current
functionalities, but all functionalities already tested in
previous iteration as well
• Reason – functionalities developed in different
iterations are not independent of each other, but rather
parts of the same software, sharing same data and
services
• Any change in data and services which some
functionality (which is currently being implemented)
uses can affect functionalities which were implemented
earlier

STUDY IN ENGLISH
Regression testing
• How do we perform regression testing?
• How to choose the tests?
• Three basic approaches:

STUDY IN ENGLISH
Regression testing
• Full regression testing (retest all)
• This is one of the methods for Regression Testing in
which all the tests in the existing test bucket or
suite should be re-executed
• In each iteration we test all functionalities
implemented in earlier iterations
• This is very expensive as it requires huge time and
resources

STUDY IN ENGLISH
Regression testing
• Partial regression testing (test selection)
• Instead of re-executing the entire test suite, it is better
to select part of the test suite to be run
• In each iteration we test just the previously
implemented functionalities which are related with the
functionalities in the current iteration
• Subset of all tests is performed
• This subset is selected in a way that we test the
modified code, and the code which could be affected
with those modification

STUDY IN ENGLISH
Regression testing
• Typically, regression testing is performed with the help
of test automation tools if possible
• Since we repeat the same tests, manual testing would
take too much time and it would be very boring
• If automation is not possible, prioritize the test cases
depending on business impact, critical & frequently
used functionalities
• Selection of test cases based on priority will greatly
reduce the regression test suite

STUDY IN ENGLISH
Regression testing
• When to do regression testing?
• Regression Testing is required when there is a:
1.Change in requirements and code is modified
according to the requirement
2.New feature is added to the software
3.Defect fixing
4.Performance issue fix (or any kind of optimization
takes place)

STUDY IN ENGLISH
Regression testing
• How do we select tests for regression?
• It was found from industry data that a good
number of the defects reported by customers were
due to last minute bug fixes creating side effects
• Hence selecting the test cases for regression testing
is an art and not that easy

STUDY IN ENGLISH
Regression testing
• Effective Regression Tests can be done by selecting the following
test cases:
• Test cases which have frequent defects
• Functionalities which are more visible to the users
• Test cases which verify core features of the product
• Test cases of functionalities which has undergone more and recent
changes
• All integration test cases
• All complex test cases
• Boundary value test cases
• A sample of successful test cases
• A sample of failure test cases

STUDY IN ENGLISH
Regression testing

STUDY IN ENGLISH
Regression testing
• Advantages of regression testing:
• It makes sure that any change (bug fix or adding new
functionality) didn’t affect the existing code
• Bugs from earlier iterations were not reintroduced
• In the most cases it can be automated
• Assurance of the product quality

STUDY IN ENGLISH
Regression testing
• Disadvantages of regression testing
• If it is performed without automation tools, it can take a
lot of time and become boring
• Even the smallest change in the code must trigger
regression testing, as it could affect already
implemented functionalities

STUDY IN ENGLISH
Regression testing
• Regression Testing Tools
• If your software undergoes frequent changes,
regression testing costs will escalate
• In such cases, Manual execution of test cases increases
test execution time as well as costs
• Automation of regression test cases is the smart choice
in such cases
• The extent of automation depends on the number of
test cases that remain re-usable for successive
regression cycles

STUDY IN ENGLISH
Regression testing
• Following are the most important tools used for both functional and
regression testing in software engineering
• Ranorex Studio: all-in-one regression test automation for desktop, web,
and mobile apps with built-in Selenium WebDriver. Includes a full IDE
plus tools for codeless automation.
• Selenium: This is an open source tool used for automating web
applications. Selenium can be used for browser-based regression
testing.
• Quick Test Professional (QTP): HP Quick Test Professional is automated
software designed to automate functional and regression test cases. It
uses VBScript language for automation. It is a Data-driven, Keyword
based tool.

STUDY IN ENGLISH
Smoke test
• Also known as “Build Verification Testing”
• It is a type of software testing that comprises of a
non-exhaustive set of tests that aim at ensuring
that the most important functions work
• The result of this testing is used to decide if a build
is stable enough to proceed with further testing

STUDY IN ENGLISH
Smoke test
• The term ‘smoke testing’, it is said, came to software testing
from a similar type of hardware testing, in which the device
passed the test if it did not catch fire (or smoked) the first
time it was turned on
• Visual inspection if the main hardware components don’t
have critical problems in circuits
• In the similar way, smoke test of the application can show
the presence of some critical failures in the application
before giving the build to the detailed testing

STUDY IN ENGLISH
Smoke test
• Smoke testing covers most of the major functions of
the software but none of them in depth
• The result of this test is used to decide whether to
proceed with further testing
• If the smoke test passes, go ahead with further testing
• If it fails, halt further tests and ask for a new build with
the required fixes
• If an application is badly broken, detailed testing might
be a waste of time and effort

STUDY IN ENGLISH
Smoke test
• Usually performed by developers, before releasing
the build to the tester, or by the testers, to verify if
the build is stable enough to continue with detailed
testing
• Usually performed with positive scenarios and valid
data
• It should always be performed before testing starts,
to confirm the app is ready

STUDY IN ENGLISH
Smoke test
• Example – Facebook-like application, consisted of
15 modules
• 4 modules out of those 15 are critical components,
such as:
• Login page
• Add user details
• Update user details
• Delete user

STUDY IN ENGLISH
Smoke test
• As part of the smoke test, we would test login page
with valid input data
• After login, we would test add user, update user
and delete user
• If all 4 critical components work as expected (with
valid input), we estimate that the build is stable
enough to proceed with testing

STUDY IN ENGLISH
Smoke test
• When to perform smoke test?
• Developers perform smoke test before giving build
to the testers
• Testers perform smoke test before starting detailed
testing
• The goal is to ensure that the basic functionalities
work ok, otherwise it doesn’t make sense to
proceed with testing cycle

STUDY IN ENGLISH
Smoke test
• Advantages:
• It uncovers bugs very early
• It exposes integration issues
• It provides some level of confidence that changes to the
software have not adversely affected major areas (the
areas covered by smoke testing, of course)
• Small number of tests and short testing time
• Can be performed by the small team

STUDY IN ENGLISH
Smoke test
• Disadvantages:
• It is not a detailed testing (not necessary a bad thing)
• It is not possible to expose all critical bugs due to a small
number of tests
• It does not include negative scenarios and invalid input
data
• Do not consider smoke testing to be a substitute of
functional/regression testing

STUDY IN ENGLISH
Sanity test
• Sanity test is very similar to the smoke test
• Smoke and Sanity testing are the most
misunderstood topics in Software Testing
• There is an enormous amount of literature on the
subject, but most of them are confusing
• Most of the testers are also not sure about the
differences between the two terms

STUDY IN ENGLISH
Sanity test
• Smoke testing is performed after software build to assure that the critical
functionalities of the program are working fine
• It is executed before any detailed functional or regression tests are executed
on the software build
• The purpose is to reject a badly broken application so that the QA team
does not waste time installing and testing the software application
• In Smoke Testing, the test cases chose to cover the most important
functionality or component of the system
• The objective is not to perform exhaustive testing, but to verify that the
critical functionalities of the system are working fine
• For Example, a typical smoke test would be - Verify that the application
launches successfully, Check that the GUI is responsive ... etc.

STUDY IN ENGLISH
Sanity test
• On the other hand, sanity test is performed on the
relatively stable application
• QA team performs sanity test after receiving a software
build, with minor changes in code, or functionality, to
ascertain that the bugs have been fixed and no further
issues are introduced due to these changes
• The goal is to determine that the proposed
functionality works roughly as expected - if sanity fails,
the build is rejected to save the time and costs involved
in a more rigorous testing

STUDY IN ENGLISH
Sanity test
• The objective is "not" to verify thoroughly the new
functionality but to determine that the developer has
applied some rationality (sanity) while producing the
software
• Hence the name – sanity test
• For instance, if your scientific calculator gives the result of 2
+ 2 =5!
• Then, there is no point testing the advanced functionalities
like sin 30 + cos 50

STUDY IN ENGLISH
System testing
• System test is based on testing the complete
system as a whole, after completing the integration
• It is defined by system requirements and system
specification
• Additionally, it includes risk-based testing, business
processes, use case scenarios and other high level
system descriptions, as well as interactions with
other systems and system resources

STUDY IN ENGLISH
System testing
• Usually it is a final test, to verify that the system as a
whole fulfills the requirements and it’s purpose
• It is often performed by testers specialists, or even
independent teams of testers
• Tests are preformed from the user perspective (end to
end)
• Since it is based on the requirement specification, it is a
black box method
• It covers both functional and nonfunctional
requirements

STUDY IN ENGLISH
System testing
• After system test, when most of the bugs are found
and fixed, system is ready for the delivery to the
client
• Client usually then initiate Acceptance Testing,
a.k.a. User Acceptance Testing (UAT)

STUDY IN ENGLISH
UAT
• Acceptance testing (UAT) is performed by the end
users, or clients
• The goal of UAT is to build the trust in the system
• Focus is on checking the functionality, by validating
that the system is adequate for the users

STUDY IN ENGLISH
Alpha testing
• Part of UAT
• One of the most commonly used testing strategies
• Testing is performed on the location of the
company which develops the software, where
programmers observe simulated users and note all
problems
• It is performed near the end of the development -
it is still possible to do minimal changes in
application design as the result of alpha testing

STUDY IN ENGLISH
Alpha testing
• Alpha testing is usually performed by a group of
independent testers (not involved in previous
iterations of testing), but within the same company
– to ensure the objectivity
• This is the last level of testing before the
application delivery
• Goal is to simulate the end users, by using black
box and white box methods, to find any remaining
error and fix it before the delivery

STUDY IN ENGLISH
Beta testing
• A.k.a. Field Testing
• Performed on the client location
• Software is delivered to limited number of real end
users, who install it and use it in real environment
• Goal – real users, when using application in real
conditions, can find errors from user perspective, which
should not be present in final version of the applcation
• A lot of companies, including Microsoft, give beta
versions to end users for testing

STUDY IN ENGLISH
Beta testing
• Advantages
• Real users get application before final delivery to general public
• Users can give feedback during beta period
• Beta testers often find new errors, from the user point of view,
which can then be fixed before global delivery
• It generally increases software quality after release to the market
• It increases customer satisfaction

STUDY IN ENGLISH
STUDY IN ENGLISH
System Testing
Miodrag Zivkovic

STUDY IN ENGLISH
System testing
• System testing is based on testing the complete
system as a whole, after integration of components
has been finished
• It is defined by functional requirements and
software specification
• Additionally, it includes tests based on the risk,
business processes, use cases and other high level
system descriptions, and interactions with other
systems and system resources

STUDY IN ENGLISH
System testing
• Usually it is a final test to verify that the system as a
whole fulfills requirements defined in specification and
its purpose
• Usually performed by tester specialists or independent
testing teams
• Tests are performed from the end-to-end perspective
• Since it is based on requirements specification, it is a
black box method
• Focus is on functional and also on non fuctional
requirements

STUDY IN ENGLISH
System testing
• System testing can be different than already
described functional testing
• It is necessary to check non functional
requirements, including system performance,
interoperability with other systems, scalability,
robustness, load test and similar
• There are over 50 different subtypes of system
testing

STUDY IN ENGLISH
System testing
Functional
System testing
Interoperability
Performance
Scalability
Stress test
Load test
Robustness
Regulatory
Regression

STUDY IN ENGLISH
Performance testing
• Performances of the system, such as response time,
reliability, resource usage and scalability are very
important
• Performance testing is a type of testing that verifies
that software system under test is behaving
adequately under expected load
• Goal is to detect and eliminate possible bottlenecks
which deteriorate system performance

STUDY IN ENGLISH
Performance testing
• Performance testing focuses on the following
aspects :
• Speed – application response time
• Scalability – detecting of the maximum user load
software can sustain
• Stability – verifies that the system under test is stable
enough under different user loads

STUDY IN ENGLISH
Performance testing
• Performance testing is actually a group of different
types of testing, out of which the most important
are:
• Load test
• Stress test
• Endurance test
• Scalability test

STUDY IN ENGLISH
Performance testing
• Applications which have bad performances (due to
poorly performed performance testing or even
complete absence of it) will generally have bad
reputation once they are on the market, and will
never achieve expected level of sales
• On the other side, critical software systems, such as
medical equipment software or software
embedded in airplanes must be additionally tested
to ensure long time operation without problems

STUDY IN ENGLISH
Performance testing
• Common problems detected by performance testing, as
expected, are application speed, response time, time
needed to load the application and bad scalability
• Speed is one of the most important application
attributes
• Slow application will almost certainly lead to loss of
potential users
• Performance testing ensures that the application
executes fast enough to attract potential users

STUDY IN ENGLISH
Performance testing
• Common performance issues (it can be noted that
the speed is a common factor for most of them):
• Long time to load the application
• Poor response time
• Bad scalability
• Bottlenecks

STUDY IN ENGLISH
Load testing
• Load testing is a type of performance testing, which
tests system performances under realistic usage
conditions
• It verifies system behavior under normal load, and also
under maximal expected load
• It identifies maximal operational capacity, possible
bottlenecks, and also which component is causing
performance degradation
• When load is increased over sensible level, load test
becomes stress test of the system

STUDY IN ENGLISH
Load testing
• Formally, load test verifies :
• Maximal operational capacity of the application
• If current infrastructure is enough for application
execution
• Application stability in case of a peak load
• Number of concurrent users which application can
sustain, and also scalability which is needed to allow
more users to the system

STUDY IN ENGLISH
Load testing
• The best example is Amazon, which organizes
Prime Day every year – global shopping day for its
prime users, with large number of discounts in a 36
hours period
• However, this event went wrong in July 2018, as
service was not available for several hours in parts
of USA due to huge amount of requests

STUDY IN ENGLISH
Stress test
• Stress tests assumes putting application under extreme
load, to observe the behavior under huge amount of
requests for data processing
• The goal of stress test is to kick application out of normal
maximal operating capacity and bring it to the point of
breaking
• While determining breaking point, we must verify if the
application limitations merit the requirements in system
specification

STUDY IN ENGLISH
Stress test
• It is important to also verify how the system breaks,
in other words to determine the shape of the
system failure
• Even in extreme load, system should not fail
completely, it should show appropriate error
message (and die gracefully)

STUDY IN ENGLISH
Stress test
• Simple example of stress test would be copying of huge
amount of text (i.e. 5GB text) to the Notepad application
• Application is in that case under stress and gives expected
error message Not responding

STUDY IN ENGLISH
Stress test
• Every properly planned stress test must focus on
the following topics:
• Verify if the system works well under extreme load
• Verify if the system shows adequate error message in
the situation where load is so high that the system can
not answer user requests
• If the system fails completely under extreme load, it can
lead to data loss, money loss and poor customer
satisfaction

STUDY IN ENGLISH
System testing
• After fixing most defects during system testing,
system is ready for delivery to the client
• Client will in most cases initiate Acceptance Testing,
or User Acceptance Testing (UAT)

STUDY IN ENGLISH
UAT
• Acceptance testing (UAT) is usually performed by
end users, or clients
• The goal of UAT is to establish confidence in the
system
• Main focus is to verify functionality of the system,
by validating if it is adequate for the users

STUDY IN ENGLISH
Alpha testing
• Part of UAT
• One of the most used software testing strategies
• Testing is performed on the location of the company
which develops software, where the programmers
observe simulated users and note problems
• It is done near the end of the application development.
It is still possible to implement minimal adjustment to
the application design as the result of alpha testing

STUDY IN ENGLISH
Alpha testing
• Alpha testing is typically performed by a group of
independent testers (not involved in the previous
phases of testing), but within the same company, to
assure objectivity of the testers
• This is the last level of testing before application
delivery
• The goal is to simulate real users, by using all
available black box and white box techniques, to
find eventual bugs and fix them before the delivery

STUDY IN ENGLISH
Beta testing
• Also known as Field Testing
• It is performed on the client location
• Software is delivered to the limited number of real
users, who install it and use it in real world scenario
• The goal is that the real users find any error from the
user perspective, under real conditions, which we don’t
want to have in the final version of the application
• Many big companies, such as Microsoft, give beta
versions to users for testing

STUDY IN ENGLISH
Beta testing
• Open i closed beta

STUDY IN ENGLISH
Beta testing
• Advantages
• Real users get the application before releasing it to the general
public
• During beta period, users can test the application and give feedback
• Beta testers often find errors which have not been noted before
that (especially from the end user perspective), and it is still
possible to fix most of them before the global delivery
• The greater the number of these errors fixed, the greater is the
software quality after releasing it to the market
• This directly increases customer satisfaction

STQA2 presentation about STQA by given university

More Related Content

Similar to STQA2 presentation about STQA by given university (20)

Recently uploaded (20)

STQA2 presentation about STQA by given university