Portfolio Project 3 - Test Plan

Portfolio Project 3
Train2raceTrain2race
Master Test PlanMaster Test Plan
Version 1.3Version 1.3
1

Train2RaceTrain2Race V 3.0V 3.0
Platform :Platform : Microsoft XboxMicrosoft Xbox
Revision HistoryRevision History
Date Author Description Version
8/1/2015 Ryan Batey Designed test plan. 1.0
12/1/2015 Ryan Batey Wrote up test matrices. 1.1
18/1/2015 Ryan Batey Finished writing test plan. 1.2
Contents
Table of Contents
2

Purpose
The purpose of this test plan is to test Train2Race in every aspect possible and
ensure it is clear of bugs and ready for release within the deadline.
The objectives of this test plan are:
• To clearly identify each section of the game which is to be tested.
• To state the most effective testing approach to carry out for each
section.
• To produce a checklist for testers to keep track of every detail during the
testing period.
I. Target quality and deadlines:
Train2Race is a triple A title, the target deadline for the testing phase is
1.5 months in duration maximum. Train2Race must be entirely tested
and ready for release within a 2 month testing period.
Game Overview
I. Setup:
Train2Race is a 3d racing game that features 10 cars that can race
around 6 race tracks. The game is single-player only, it has no network or
multiplayer modes.
Supported Platforms: Microsoft Xbox Only
Supported Controller: Standard Xbox Controller
3

Only
Player Capabilities: 1 Player
Multiplayer: Not Supported (including
network mode)
Age Rating: All, Everyone
Languages: EFIGS
Territories: TBD, provisionally UK, US
and all other EFIGS
territories
II. Pick Ups:
A range of power ups/downs are scattered throughout each track. These
are in the form of icons on the track that must be driven over to collect
them. Items comprise:
Extra Fuel – adds 10 to the fuel counter (which goes from 0-100 internally).
Turbo boost – adds 10 to the turbo boost meter (which goes from 0-100
internally).
Oil slick – reduces traction for a time, potentially bad for handling.
Spanner – repairs any and all damage sustained by the car so far, including
damaged tyres.
Tacks – causes punctures, permanently degrading traction. Can only be
repaired by the spanner pickup.
Missile – immediately launches a missile from the car picking this up and
that flies straight ahead until it hits something.
Each track should feature at least one of each item.
III. Front-End
4

The front-end is rudimentary and only has options for single race, campaign
and settings. The hierarchy is as follows:
IV. Single race
- Race selection screen
- Track selection
V. Campaign
- Option to resume campaign (if it was aborted early) or start new, otherwise
straight into game.
VI. Settings
Difficulty: easy, medium, hard.
Controller: left-handed, right-handed.
Language selection: EFIGS (via national flags).
Each option screen has a back/cancel option.
VII. Scoring
In single-race mode, ranking is based purely on race position. In campaign
mode, points are awarded according to the position attained in each race. The
winner of the campaign is the racer with the most points. Points are
10,6,4,3,2,1 for positions 1,2,3,4,5,6 respectively. Between races in campaign
mode, there is a single rankings page to display the current ratings.
VIII. Load/Save
There are no explicit load or save commands. Progress cannot be saved during
a race; choosing to exit a race will lose all information/track position etc. In
campaign, career progress to date and the track about to be raced is
automatically stored however. When this data is loaded (by resuming a
campaign), the player will have the same ranking as before and be placed at
the start of the track currently being played.
IX. In-game
There are no in-game options, though when the Start button is pressed,
a pause menu appears and the game is halted. Options to return to the
main menu, restart current race and resume racing are offered.
Resources:
5

For more information on Train2Race,
See pdf document - project_files-GT_portfolio_project_03.
Debug Requirements:
• Only standard debug kits are to be used by each tester when
testing the game.
• Each tester is to use standard retail versions only.
• Cheats are not to be carried out when testing.
Test Team/Contact-Info/Responsibilities
Role Tester
Name:
Contact Details:
Mark Jones
Tel: 0800 123 124 Email: Mjones@qatesting.com
Overview Mark will be the main functionality tester of the
project, he will be testing single player as assigned by
the test matrix. Each tester is to work in house, and
closely with the developers to resolve issues quickly.
Responsibilities Single player testing as assigned by the test matrix
Bug reporting using the internal client
Hardware PC#001 Microsoft Xbox #001
Location Liverpool, UK
6

Role Tester
Name:
Contact Details:
John Smith
Tel: 0800 123 125 Email: Jsmith@qatesting.com
Overview John is the projects secondary functionality tester. He
will mainly be carrying out soak tests and play
through tests. He will also be testing achievements
and unlocks. Each tester is to work in house, and
closely with the developers to resolve issues quickly.
Responsibilities Soak testing
Play through testing as assigned by the test matrix
Achievements and unlocks
Role Tester
Name:
Contact Details:
Robert Johnson
Tel: 0800 123 126 Email: Rjohnson@qatesting.com
Overview Robert is the project’s third functionality tester, he is
to test the front end and pause menu of Train2Race.
Each tester is to work in house, and closely with the
developers to resolve issues quickly.
Responsibilities Front end and pause menu as assigned by the test
matrix
Role Hardware Compatibility Tester
Name: Denise Rogers
7

Contact Details: Tel: 0800 123 127 Email: Drogers@HCTesting.com
Overview Denise is a third party hardware compatibility tester
and will be testing in house with the rest of the test
team. She works for a third party testing group that
specialises in hardware compatibility testing and will
act as the point of contact with them. Denise will be
assigned to the project until the testing phase is
complete, her main workload is on localised versions
of the title to provide a more complete all round
localisation testing phase.
Responsibilities Hardware compatibility testing
Hardware The company Denise works for have provided a suite
for her to test in and set it up in house at the
developer.
Role Compliance Tester
Name:
Contact Details:
James Andrews
Tel: 0800 123 128 Email: Jandrews@Comptesting.com
Overview James is a third party compliance tester who will be
working in house with the test team. He is to ensure
all technical requirements are compliant with the
hardware manufacturer’s guidelines, mainly the
official racing body’s requirements as they have right
of approval on the finished game. James is to test in
house until the testing phase is complete, to ensure
all requirements are met legally.
Responsibilities To ensure the game is compliant with the hardware
manufacturer’s guidelines and all requirements are
legally met.
Bug reporting using the internal client.
Hardware Microsoft Xbox #004, #005, #006 and #007.
PC#005
8

Assumptions and Dependencies
The time to turn around each test round will each be a 5 day period at
most. Depending on how large an issue is, extra time will be allowed for
testers to work with the developers to resolve issues quickly.
Each build should be delivered in as short a time period as possible to
ensure the game is ready for release within the time schedule.
Project Risk Analysis
Each assigned tester is to be aware of the risks of the project while
testing the game. Testers should be aware that the game is to be
released to the public with least amount of issues as possible, so that it
is to meet the requirements of the developers. These risks include;
• Is the game fun to play?
• Is the game playable as a whole from the
consumer’s point of view?
• Does the game meet expectations as it was shown
in trailers previous to release?
• Does the game include factors to keep the
audience entertained all the way through?
All of these are questions each tester should commonly ask themselves
while the testing phase is being carried out. Each tester is to ‘put
9

themselves in the consumer’s shoes’ so that the game will meet the
developer’s expectations.
Testing Priorities and Focus
All testing is mainly being focused on functionality testing/feature
testing for the testing phase. Each tester is assigned to methods of
testing by a test matrix for an overall more effective testing phase of the
entire game. Play through testing and functionality testing/feature
testing are to be covered as thorough as possible by the test matrix in
order to save time effectively. The game has been broken down into
sections so each tester can navigate the test plan easily.
The sections are based on single-player, play through and front end and
pause menu. Installation testing and acceptance testing is to be carried
out by all testers on the title before all main testing is initiated via the
test matrices. Localisation testing training will be provided to all testers
before the testing phase is underway and each tester is to use the
matrices in the test plan to carry out localisation testing where
applicable, during the main testing phase. After all of the main testing
has been covered, configuration tests, verification tests and regression
tests are to be carried out on the title by all testers.
Scope and Limitations of Testing
Each type of testing above is to be carried out within a 1.5 month
period. The main focus is so that the play through testing and
functionality testing/feature testing phases are carried out specifically,
however as long as these types of tests are covered as much as possible
soak testing can be carried out. Extra testing will cover much more of
the project and lead to a better carried out test plan to ensure quality is
assured for the developers.
Testing is limited to a lack of console testers within the company, to
tackle this issue we will be using a third party compliance tester and also
a third party hardware compatibility tester to help to complete the
testing phase more efficiently. The hardware compatibility tester is to
provide more efficient overall testing for localisation testing, which will
lighten the work load and in house testers can focus on testing assigned
by the matrix.
10

Test Outline
I. Installation Testing
II. Acceptance Testing
III. Functionality Testing/Feature Testing
IV. Level Checklist
V. Single Player
VI. Front End and Pause Menu
VII. Other - Soak Tests, Play Throughs, Achievements
and Unlocks
VIII. Configuration Testing
IX. Verification Testing
X. Regression Testing
I. Installation Testing
This is the first test to be run on a build, this type of testing must be done by
the lead tester/burn mastering technician before duplicating and then passing
to the team for further testing. This type of testing should be carried out on
every build of the game and run on all relevant platforms. Instructions for
testing are as follows –
• Insert the disc in to the Xbox system.
• If the game boots, it has passed.
• Try to manually boot the game, if it does not boot then it has failed the
test.
So long as the game passes the installation tests, acceptance tests can then be
carried out on the title.
I. Acceptance Testing
Acceptance tests are to be carried out to check whether the game is stable
enough to be completely tested in QA. This test should be run on every build,
11

this type of testing should be carried out immediately after the installation
tests are completed and should take no more than thirty minutes to complete.
Acceptance testing is to test each section of the game is functioning correctly,
the acceptance test is failed if the following occurs –
• Failure to properly load or execute.
• An excessive number of crashes, choppy video or sound or excessive
graphical problems.
• An excessive number of errors that may lock out parts of the game from
testing.
The testing sequence is short and is to verify the game is stable, it is to be
carried out by the lead tester as follows –
 Test the game loads up to the main menu.
 Test all menus load up - settings, difficulty, controller settings and
language selection.
 Test single player campaign loads up and the game can be played.
 Test single race loads up correctly, through the race selection
screen and then track selection screen.
 Test all sound plays correctly.
 Test all video plays correctly.
Once the game has passed the acceptance tests, the game can be passed to
the testing team to carry out proper testing on the title.
I. Functionality Testing/Feature Testing
II. Level Checklist
Level
:
Tested
:
Not
Tested
:
In
Progress
:
Assigne
d
Tester:
Estimate
d Time:
Actua
l
Time:
Instructions
:
Comments
:
Level
1
Level
2
Level
12

III. Single Player
Note: all 6 levels are to be tested in the same manner as the matrix dictates.
Level Tested Tester Comments Added Test Instructions
Visual
Are the textures
displayed correctly?
Are all animations
playing correctly?
Any flickering between
the textures?
Are all textures blended
correctly?
Is there any overlap
between graphics?
Are all graphics
displayed properly when
pickups are acquired?
Are all graphics
displaying correctly
when overtaking?
Are graphics displaying
correctly during a
collision?
Text
Is all text displayed on
screen correctly?
14

Is text displayed in the
correct selected
language on screen?
Does all text match the
correct terminology as
used in gaming?
Is all spelling and
grammar on screen
correct?
Are there any words
displayed in an incorrect
manor?
Does all text have the
correct spelling used for
each term used in the
game?
Does text appear on
screen and disappear off
screen correctly?
When text is displayed is
there any lag or glitching
at that specific time?
Audio
Does all music play
correctly?
Is there any distortion to
music when SFX are
played?
Are all SFX played
15

correctly?
When more than one
event at a time is carried
out are the sound clips
playing thoroughly?
Does each song loop
properly when it’s
finished?
AI
Does the AI react to the
player correctly?
Whilst playing through is
the AI behaving as it
should?
As a collision happens,
does the AI collide
correctly?
As an event happens, is
the AI glitching in any
way?
Camera
Does the camera
function correctly when
the player attempts to
use it?
Is the camera reacting to
the player’s movement
correctly?
Does the camera glitch
at all when an event
occurs?
Is the movement of the
16

camera smooth?
Is there any sticking of
the camera when it is in
motion?
Does the camera effect
the player’s ability to
see what is happening as
the game is played?
Options
Does the options menu
appear correctly on
command?
Are all the consistent
menus available within
the options menu?
Is it possible to navigate
the options menu with
ease?
Does bringing up the
options menu effect
gameplay at all?
Within the options
menu, do all settings
work correctly?
Is there any lag in the
options menu when in
use?
Cut-scenes
Do the cut-scenes play
through smoothly?
Does a cut-scene
initiating effect
17

gameplay?
Can cut-scenes be
skipped correctly?
Is there any lag when a
cut-scene is played?
Do cut-scenes cause the
game to crash?
Are the textures in cut-
scenes smooth?
Level specific events
(progression)
As the player gains a
place on the grid, does
the score improve
respectively?
Does gaining a place on
the grid force any UI
errors?
Are there any graphical
errors as a place is
gained?
When the player
overtakes, does the
gameplay flow
smoothly?
As a pickup is collected,
does the UI show any
errors?
Do all pick-ups work
respectively?
When pick-ups are
engaged, does gameplay
18

flow smoothly?
Does collecting a pick-up
cause any graphical
errors?
As the player earns a
new place on the grid, is
the score adjusted
respectively?
Does the score cause
any UI errors as a new
score is introduced?
Is gameplay running
smoothly as a new score
is achieved?
Does achieving a new
score effect the games
graphics?
Does achieving a new
score effect AI correctly?
As the end of the race is
met, does the UI
respond correctly to
scoring?
Is there any graphical
errors as the end race
scene is played?
Does the gameplay run
smoothly as the race is
completed?
Is there any obstructions
to the race scene playing
in full?
19

Boundary Checking
When more than one
button is pressed at a
time, does the game
freeze at all?
Pushing a button by
mistake, does the game
react effectively?
When a button is
pressed over and over
again, is the game
caused to freeze?
Does driving towards
the barriers/off map
cause the game to
glitch?
Turning around
immediately as the race
starts, does the game
react properly?
Driving into any other
car continuously, does
the game glitch at all?
Driving against the
barrier, does the player
get stuck?
Constantly crashing into
the same place on the
map, does the player car
restart in place
properly?
Forcing other cars to
20

crash, does the game
react correctly?
Pushing every button on
the controller at once,
does the game react
correctly?
Objects/scenery
Are all objects on the
map lined up smoothly?
Is there any blur in
textures on the scenery?
Are there any gaps
between objects and the
scenery?
When crashing are the
graphics on the scenery
affected negatively?
Can the player interact
with the scenery
correctly?
Does the AI affect the
scenery negatively when
interacting with objects?
Do all pick-ups work
correctly when
acquired?
Is there at least one of
each pickup on each
map?
Does using a pickup
cause the game to jump
or lag?
21

Do pick-ups force the
game to run slower?
IV. Front-End and Pause Menu
Note: each tester is to use the above level checklist where necessary.
Front-End Tested Tester Comments Added Test
Instructions
Does the single race menu appear
correctly when selected?
Is there any lag whilst navigating
menus?
Does track selection appear
correctly when selected?
Is it easy to navigate front-end
menus?
Does selecting the menus cause
any glitch to the game?
Does the campaign menu appear
correctly?
Does selecting campaign cause a
glitch to the game?
Does selecting resume campaign
22

appear correctly?
Does selecting start new appear
correctly?
Does each selection work as it
should once selected?
Settings
In the difficulty menu, is each
difficulty selectable respectively?
Can difficulty be changed with
ease?
Does choosing a difficulty cause
any errors in the menu selection?
Can the controller selections be
changed with ease?
When selecting a different
controller scheme, do the controls
change respectively?
Does each control scheme affect
the menus being navigated
negatively?
In the language selection menu,
can each language be selected with
ease?
Does the language change correctly
when a new language is chosen?
Are all words on screen in the
correct language as chosen?
On each screen, does the back
button work correctly?
Does pressing the back button
cause any graphical errors?
23

As the back button is pressed does
the game freeze?
In Game Options
Does the game pause correctly
when the button is pressed?
Does pausing the game cause any
lag?
When the game is paused does any
freezing occur?
Does pausing the game affect
gameplay negatively?
Does pressing return to start menu
work correctly?
When return to start menu is
selected, does the game freeze?
Is there any lag when returning to
start menu?
Does pressing restart current race
cause any freezing?
Does restarting current race affect
the gameplay negatively?
Does the game load correctly when
restart current race is selected?
When restarting current race, do all
scores and positions reset
correctly?
When resume race is selected, does
the game have any lag?
Does selecting resume race
continue the game correctly?
Is there any freezing when resume
race is selected?
24

Does resuming the race affect the
score negatively?
Load/Save
Does the game save correctly?
Does the game freeze when saving?
Does saving cause the game to lag?
Does loading the game work
correctly?
When loading the game is there
any lag?
Is the game being loaded to the
correct save file?
When the game has been loaded,
was the game saved at the correct
place when it was last saved?
Does loading/saving cause the
game to function incorrectly?
25

V. Other – Soak Tests, Play-throughs, Achievements and Unlocks
Achievements and Unlocks Tested Tester Comments Added Test
Instructions
Once a race is won, is the next race
unlocked respectively?
Does the next race unlock when the
race is lost?
Does the next race unlock when lower
points (than are required to unlock the
next race) are achieved in the race?
When the player gains a position in the
race are the correct points added?
As a new position is gained, is game
flow smooth?
Is there any lag when taking a position
in the race?
Does gaining a position have a negative
effect on the opposition?
When winning the race, are points
added correctly?
Does winning the race cause the game
flow to jump or lag?
Is the opposition reacting negatively
when the race is being won by the
player?
As the player wins the race, do all
assets work respectively?
When the player wins the race, is the
26

score adjusted respectively for the
opposition?
Playthroughs
Playing the game as a customer would,
do all assets seem in place?
Does the game flow smoothly with no
bugs?
Is it possible to complete the game?
Can all scores be achieved?
Is the game attractive to play?
Does the game run as it should and play
properly?
Playing the game to win, do all scores
stack respectively?
Is it possible to win each race in first
place?
Does the game finish correctly when
the player wins every race?
Playing the game to lose, does the
game finish correctly?
Do scores stack properly when losing?
Does losing effect the game flow
negatively?
Playing the game and finishing in any
other position than 1st, does the game
play through properly with no bugs?
Soak Tests
Does the game slow down after being
run through the demo mode for a
27

number of hours?
Does running the game through a soak
test cause any performance issues?
Is there any memory leaks due to
running a soak test?
Does the console stall after a soak test
is run?
Is response time in all functions
functioning correctly?
Is the game crashing after running a
soak test?
Does the game load up correctly after a
soak test has been run?
Does the game install correctly after the
soak test?
Is the game saving/loading correctly
after a soak test?
Are all buttons functioning correctly?
Do all menus navigate smoothly?
Do all cut-scenes run smoothly?
28

VI. Configuration Testing
Configuration testing consists of running all of the functionality tests that have
been used in the test plan to ensure proper functionality on Microsoft Xbox.
The test should be run on beta and master candidate discs, testers must use a
copy of the functionality test plan to ensure –
• All graphics should be tested to verify that they fit within the TV safe
area.
• Ensure that the software is tested on all known firmware and hardware
releases of the console e.g. all models of Microsoft Xbox. This also
includes specific debug machines.
I. Verification Testing
The purpose of verification tests is to verify that bugs assigned as FIXED, are
indeed fixed. Verification tests should be run on every build except those
where no bugs are claimed as FIXED. The verification testing is to be executed
in this fashion –
• The lead tester will open the QA database and filter all bug reports with
the status flagged as FIXED.
• The lead tester will then print them out and sort all bug reports by
grade, priority, category and the original tester’s name.
• The bugs are then separated by tester’s name and given to the
appropriate tester for verification.
• Bugs are to be re-tested by the assigned tester for each bug report.
• The bug database is to be updated assigning bugs as VERIFIED FIXED or
REOPEN.
I. Regression Testing
29

Regression testing is to be carried out regularly, generally once every three
builds. It should also be done with all master candidate builds. All testers are
assigned to regression testing to verify all issues found in the testing cycle are
and have remained FIXED. The primary types of bugs that will need to be
tested include: CLOSED, FIXED and REOPEN. Regression testing is to be
executed in this manner –
• The lead tester is to print out all bug reports to be regressed.
• Separate bug reports by type and tester name. Sort bug reports by
grade, priority and category.
• Give them to the appropriate tester. Grade ‘A’ bugs are to be tested
first.
• The lead tester should then collect all records after they are complete
for entry in to the database.
• Update the QA database.
Test Environment
• Testing is to be carried out on standard Microsoft Xbox consoles
only, using a standard Xbox controller.
• All testing is to be in house at the developer, each tester is to
write up bugs into the internal bug reporting client.
• Only standard debug kits are to be used by each tester during the
testing phase, in certain testing circumstances.
• Using standard consoles will ensure the game is tested to a
consumer’s point of view, no debug consoles are to be used as
they may have extra memory or a faster processor.
Patching/Build Deliveries
As a new build is released to be tested, each tester is assigned to use the test
matrices as a guide to thoroughly cover each build in turn. Testers will be
30

notified when a new build is integrated into the system, the bug database will
close the older build issues making testing much quicker and saving time.
Configuration testing is to be run only on beta and master candidate builds,
verification testing is to be run on every build except those where no bugs are
claimed FIXED. Regression testing is to be carried out regularly on every three
builds and should also be run on all master candidate builds. All other types of
testing are to be run on every new build released.
Bug Database Setup
The bug database will be the standard reporting client and setup for reporting
bugs, the database program is Mantis and will be connected to a server in
house for all computers to connect to with ease. Bugs will be grouped into
Unassigned, Recently Modified and Resolved groups. Bugs are colour coded
according to status e.g. new – red, acknowledged – yellow and resolved –
green, each bug will have a unique number that is used to identify issues.
When the user clicks the bug number it brings up extended information about
that bug, all attached files will be shown such as; screen shots and videos taken
by the tester. The user is able to add information to the system about the bug,
apply a new status and so on. As a bug is assigned as Fixed by a programmer
the tester who originally found the bug will be emailed with its status.
I. Outline of Bug Reporting
To report bugs each tester is to follow a path of instructions and use the stated
programs to describe the bugs clearly.
• Use Fraps to take screenshots and videos of bugs.
• Use Windows Movie Maker to adjust videos for easier pin-pointing of
the bugs.
• Testers can also use a HD video camera to film gameplay where
applicable.
• Microphones can also be used to narrate if the issue isn’t clear enough.
Here is an example how bugs should be reported in list format, the below
template is to be used by each tester:
31

Assign a bug ID to each bug to keep order for the database, add the intended bug category for each
bug below.
Bug ID Bug A Project Train2Race Category UI
Description Write the bug description in short here –
(example: show steps to reproduce the bug)
Steps to reproduce:
1.
2.
Write the steps up in list format as above.
State how many times the bug has been reproduced and what percentage the bug
reproduces at here.
Observed
Result
State the observed result of what happens.
Expected
Result
State the expected result of what should happen.
Related files (Any video clips or screen shots should be stated here).
I. Reporting Requirements
All bugs are to be written up in the above list format. Once a tester has written
the bug report, the tester is to send the report to the developer via email
address. Each tester will have the phone number and email address of the
developers if any issues need to be resolved quickly and at hand.
Debug Kits and Automation
Only standard debug kits are to be used by each tester, each tester will be
running their own personal PC with the debug kits running continuously.
Debug kits must keep running for the programmers to sieve out bugs and fix
them as required.
No automated testing is to be carried out, all testing is to be done manually to
save money and time efficiently.
32

Problem Tracking and Resolution
The problem tracking and resolution process will be carried out as
follows:
 As soon as a bug is reported and enters the database it is set to
NEW and is assigned to the lead tester automatically.
 Once the lead tester reviews the bug it is set to OPEN and is then
assigned to the producer.
 Now the producer assigns the bug to the most relevant member
of staff based on the bug type.
 The developer who is assigned to the bug has five choices;
1. NEED MORE INFO – will be stated and the bug will be
assigned back to the original tester if more
information is needed. The tester will then be able to
REOPEN the bug.
2. A FEATURE – if the developer believes the bug is a
non-issue they may claim the bug as the above. The
tester will once again be assigned the bug and decide
to state the bug as CLOSED or to report the bug
through the chain of management. The Lead tester,
QA manager, producer then development director (if
the previous cannot decide) will decide whether to
state the bug as FIXED, then back to the tester to
review the bug as VERIFIED FIXED.
3. NO FIX – if the developer decides the bug cannot be
fixed the above will be applied and the bug will be
sent back to the tester. The tester can decide
whether to assign the bug as CLOSED or to escalate
the bug through management and follow through the
above steps to state the bug as VERIFIED FIXED.
4. DESIGNED THAT WAY – if the developer decides to
state the bug as the above choice, the process as
stated before (through chain of management) will be
carried out if the tester decides differently. The bug
will be assigned to the designer, and the designer will
make changes to the design document and then
assign the bug to the developer to be stated as FIXED.
33

Once again the bug will be assigned back to the tester
to be reviewed as VERIFIED FIXED.
5. FIXED – once a developer fixes a bug the bug is stated
FIXED and assigned back to the tester to review the
bug. If the tester decides it is not fixed then the bug is
stated REOPEN and assigned back to the developer. If
the bug is FIXED the tester will state the bug as
VERIFIED FIXED and the bug will not be assigned to
anyone else, it will be at the end of its life cycle.
I. Severity and Grade of Bugs
Grade A: a major bug that would prevent the game from being approved or
published by QA.
• A bug that closes or blocks off areas of the game
• Any bug that causes the machine to lock up, crash, reset the machine or
renders the controller inoperable.
• Any hardware compatibility problem that prevents the game from being
run on the system requirements.
• All copyright and trademark issues.
• Content that could contravene the targeted age classification rating.
• Any major problem encountered during the installation process that
prevents software from being installed successfully.
• Non localisation in localised versions in main areas of the game.
Grade B: a serious logic or gameplay defect.
• Gameplay or UI problems that would hinder the player’s enjoyment.
• Serious graphical corruption or missing artwork.
• Localisation spelling errors, such as missing special characters.
• Severe game logic problems that would become obvious to the player.
• Serious problems caused by the implementation of localised text, audio
or graphics.
• Audio problems encountered during playback or due to implementation.
• Installation configuration and set-up problems.
34

• Localised versions still having non-localised language.
• Grammar or spelling mistakes.
Grade C: minor bugs which should be fixed, but is not essential for
functionality and release of product.
• Incorrect implementation of minor functions or features.
• Obscure game logic problems that don’t cause any side effect (that can
be played around).
• Inconsistencies between the manual and the software.
• Out of context or inappropriate phrasing.
Grade D: gameplay suggestions or improvements recommended by QA
that would enhance the products overall quality.
• Suggestions to improve gameplay or interface.
• Additional game design suggestions.
• Unintentional game features that would not be noticed by the player
unless they had read the original design document.
• Issues for the ‘read me’ file.
I. Priority
The order in which issues need to be fixed:
• High
• Medium
• Low
Software Entrance and Exit Criteria
Software is to be tested at the beta, master candidate and alpha stages,
on new builds testing will start with quick sanity tests being run. After
the previous phase of testing, Train2Race is to be tested overall using
35

the black-box testing strategy. Beginning with installation tests and
acceptance tests then on to functionality testing, play-through testing
and soak testing. Configuration tests, verification tests and regression
tests shall then be carried out, once all above testing is finished then the
testing phase is complete.
Personnel Pre-Training Needs
Localisation testing training will be provided to all in house console
testers before the testing phase is under way. We will be enlisting the
help of a third party hardware compatibility tester to lighten the load for
in house testers. The hardware compatibility tester specialises in
localised games testing and will be assigned most of the localisation
testing during the testing phase.
Sanity Testing Period
Sanity tests are to be carried out by all three testers when a new build is
released to quickly cover the new build and ensure there are no issues
left from the previous build.
Sanity tests are to be suspended when all previous builds are checked
and tested by each tester previous to the black box testing phase.
Security and Licensing
There is a non-disclosure agreement each tester is urged to sign for
Train2Race, no information is to be leaked about the game in
development and it will be classed as an offence to do so.
Microsoft’s Technical Certification Requirements are to be followed to
ensure quality and compatibility of Train2Race with Microsoft Xbox
consoles.
The third party compliance tester is to ensure the game complies within
all of the above rules and especially the official racing body’s rules for
the title as they have right of approval over the finished game.
Glossary/Acronyms
36

3d – Three dimensional, an object that has height, width and depth, like
any object in the real world.
Acceptance Testing - acceptance testing is a test conducted to determine
if the requirements of a specification or contract are met.
Ai – Artificial Intelligence, a branch of computer science dealing with
the simulation of intelligent behaviour in computers. The capability of a
machine to imitate intelligent human behaviour.
Alpha – Alpha is the stage when key gameplay functionality is
implemented, and assets are partially finished. A game in alpha is feature
complete, that is, game is playable and contains all the major features.
Analysis - Detailed examination of the elements or structure of
something.
Assets - Assets are classified as images, multimedia and textual content
files.
Automation – An automatic state of testing, carried out by computers
rather than testers.
Black box testing - Black-box testing is a method of software testing that
examines the functionality of an application without peering into its
internal structures or workings. This method of test can be applied to
virtually every level of software testing: unit, integration, system and
acceptance.
Boots – When a game loads up on a game system correctly.
Bug – A defect in a games software or assets.
Bug database – The database that information about bugs is stored in
once a game has been tested and bugs are reported by testers.
Bug reporting – The approach testers take to write up bugs once a game
has been tested.
Bug reporting client – The software used by testers to report bugs.
Build – A version of a game that is released to users.
Campaign - A campaign setting is usually a fictional world which serves
as a setting for a role-playing game or war game campaign.
A campaign is a series of individual adventures, and a campaign setting
is the world in which such adventures and campaigns take place.
Cheats – A number of button presses or codes that a player can type in
so they have numerous ways to exploit a game to their advantage.
37

Checklist - A list of items required, things to be done, or points to be
considered, used as a reminder.
Choppy – Rough or distorted.
Collide – When two or more objects come into conflict.
Collision – See collide.
Configuration Testing - Configuration testing is the process
of testing the system with each one of the supported software and
hardware configurations.
Contact – Details of a person who is involved in the game test plan.
Crashes – When a game freezes and no input by the player changes what
the game is doing.
Cut-scenes – A series of videos played through-out the game.
Debug - The process of identifying and removing errors from computer
hardware or software.
Demo - Demonstrate the capabilities of (software or another product).
Developer – The person or people that have developed the game.
Duplicating – To create a second copy of an original.
Event – When something important happens, like a player gaining a
position in a race.
Feature – Something featured in the game such as a pickup.
Fraps – A program used by testers to record gameplay and take
screenshots.
Freeze – When a game locks up and is known as “frozen”.
Front-end – The main front page of a game that includes all modes to be
played and options/settings.
Functionality – How the game functions and what functions involve.
Gameplay – How the game generally plays through, usually triggered by
a person playing the game.
Glitching – When a game suffers a sudden malfunction or fault.
HD – High definition graphics.
ID – Identification.
Implementation - The process of putting a decision or plan into effect;
execution.
38

Info – Information.
Installation Testing - Installation testing is a kind of quality assurance
work in the software industry that focuses on what customers will need
to do to install and set up the new software successfully.
Interface - A device or program enabling a user to communicate with a
computer.
In-game – Things that happen while a game is being played are classed
as in-game.
Integrated – With various parts or aspects linked or coordinated.
Lag – When a game slows down, jumps and skips unintentionally.
Leaked – When information is given out without lawful consent.
Level - A level, map, area, stage, world, rack, board, or zone in a
video game is the total space available to the player during the course of
completing a discrete objective. The term "level" can also refer to
difficulty level, as in a degree of difficulty.
Load – A setting in a game that can initiate a previous game save.
Localised – See localisation.
Localisation - Game localisation refers to the process of transforming
videogame software and hardware for preparation to be imported and
sold in a new region, usually a different country.
Mantis – Bug tracking software.
Manually - defined as something done by hand or using your physical
labour as opposed to something done using a machine.
Map – The area of a level in a game.
Master – The finished product of a game/test plan.
Matrices – Charts and tables used in the test plan.
Matrix – See matrices.
Microsoft – The leading company of the hardware the game is to be
tested on.
Multiplayer - Denoting a computer game designed for or involving
several players.
Network – A mode for playing the game over the internet with other
players.
39

Non-disclosure agreement - A contract by which one or more parties
agree not to disclose confidential information that they have shared with
each other as a necessary part of doing business together.
Options – A menu in which all settings of a game can be changed.
Outline - A general description or plan showing the essential features of
something but not the detail.
Patching - A patch is a piece of software designed to update a computer
program or its supporting data, to fix or improve it.
Pausing – When a player presses a button to bring the game to a
temporary halt.
PC – Personal computer.
Platform – The system that is used to play the game (Xbox).
Play-through - A more relaxed and casual focus on the game that is
being played, as the tester plays the game through from start to finish.
QA – Quality assurance.
Regression Testing - Regression testing is a type of software testing that
seeks to uncover new software bugs, or regressions, in existing
functional and non-functional areas of a system after changes such as
enhancements, patches or configuration changes, have been made to
them.
Sanity testing - Software sanity tests are synonymous with smoke tests.
A sanity or smoke test determines whether it is possible and reasonable
to continue testing. It exercises the smallest subset of application
functions needed to determine whether the systems are accessible and
the application logic is responsive.
Save – A setting a player can select to save progress of a game.
SFX – Sound effects.
Score - The number of points, goals, runs, etc. achieved in a game or by
a team or an individual.
Scoring - Gain (a point, goal, run, etc.) in a competitive game.
Single player - A single-player video game is a video game where input
from only one player is expected throughout the course of the gaming
session.
Single race – A setting where the player can choose to do a single player
race.
40

Soak tests – See sanity testing.
Stable – A build of the game that does not break easily.
Test - Take measures to check the quality, performance, or reliability of
(games), especially before putting it into widespread use or practice.
Tested – When a test has been run on a game, the tester assigned should
state the game as tested.
Testing – See test.
Testing phase – The length of time testers test a game for, usually a strict
number of set hours or days.
Test plan - A test plan is a document detailing the objectives, target
market, internal beta team, and processes for a specific beta test for a
software or hardware product. The plan typically contains a detailed
understanding of the eventual workflow.
Textures – The texture of graphics in the game.
TV Safe Area - A term used in television production to describe the areas
of the television picture that can be seen on television screens.
UI – User interface, the means by which the user and a computer system
interact, in particular the use of input devices and software.
Verification Testing - Verification is the process of checking that a
software system meets specifications and that it fulfils its intended
purpose. It may also be referred to as software quality control.
Version/V – A particular form of something differing in certain respects
from an earlier form or other forms of the same type of thing.
Windows movie maker – A program testers use to record and edit video
of a game that is being played.
Xbox – The hardware the game is run on.
41

Portfolio Project 3 - Test Plan

More Related Content

Similar to Portfolio Project 3 - Test Plan (20)

Portfolio Project 3 - Test Plan