chapter 6.pdf Software Configuration, Quality Assurance and Maintenance

6.1 Risk Analysis & Management: Risk Mitigation, Monitoring and
Management Plan (RMMM).
6.2 Quality Concepts and Software Quality assurance Metrics,
Formal Technical Reviews, Software Reliability
6.3 The Software Configuration Management (SCM) ,Version Control
and Change Control
Chapter 6 Software
Configuration Management,
Quality Assurance and
Maintenance
By Megha V Gupta, NHITM

Risk Analysis and Management
⚫ Risks are potential problems that might affect the
successful completion of a software project.
⚫ Risks involve uncertainty and potential losses. Risk analysis
and management are intended to help a software team
understand and manage uncertainty during the
development process.
⚫ The important thing is to remember that things can go
wrong and to make plans to minimize their impact when
they do.
⚫ The work product is called a Risk Mitigation, Monitoring,
and Management Plan (RMMM).

Risk Strategies
⚫ Reactive risk strategies:
o Very common, resources are set aside to deal with likely risks, should they
become a problem.
o The software team does nothing about risks until something goes wrong.
o At that point the team must work quickly to resolve the problem.
⚫ Proactive risk strategies:
o Risk management begins before technical work starts.
o Risks are identified, their probability and impact are assessed and they are
ranked by importance.
o The team builds a plan to avoid risks if they can or minimize them if the risks
turn into problems.

“Risk& Reward” “Plan for the risk” to earn a “reward”

Characteristics of Software Risks
⚫ Uncertainty: the risk may or may not happen; that is
there are no 100% probable risks. (A risk that is
100% probable is a constraint on the software
project.)
⚫ Loss: if the risk becomes a reality, unwanted
consequences or losses will occur
⚫ When risks are analyzed it is important to quantify
the level of uncertainty and the degree of loss
associated with each risk.
Developed by Reneta Barneva, SUNY Fredonia

Software Risks
Software Risks (Approach 1)
⚫ Project risks
⚫ Technical Risks
⚫ Business Risks
Software Risks (Approach 2)
⚫ Known Risks
⚫ Predictable/Unpredictable Risks

Project Risks
⚫ Threaten the project plan.
⚫ Usually result in delays in project schedule
and increased costs.
⚫ Identifies potential budgetary, schedule,
personnel, resource, customer, and
requirements problems and their impact
on a software project.

Technical Risks
⚫ Threaten product quality and timeliness of
the schedule.
⚫ Identifies potential design,
implementation, interface, verification,
and maintenance problems.

Business Risks
⚫ Threaten the viability of the software to be built.
⚫ Top 5 business risks:
1. Building an excellent product or system that no one
really wants (market risk).
2. Building a product that no longer fits into the overall
business strategy for the company (strategic risk).
3. Building a product that the sales force doesn’t
understand how to sell (sales risk)
4. Losing the support of senior management due to a
change in focus or a change in people (management
risk).
5. Losing budgetary or personnel commitment (budget
risks)

Known Risks
⚫ Known from careful evaluation of current
project plan.
⚫ Examples: Unrealistic delivery date, lack
of documented requirements or software
scope, and poor development
environment.

Predictable/Unpredictable
Risks
⚫ Predictable Risks are extrapolated from
past project experience.
⚫ Examples: staff turnover, poor
communication with the customer,
dilution of staff.
⚫ Unpredictable risks are extremely difficult
to identify in advance.

Seven Principles of Risk Management
⚫ Maintain a global perspective
– View software risks within the context of a system and the business
problem that is is intended to solve
⚫ Take a forward-looking view
– Think about risks that may arise in the future; establish contingency
plans
⚫ Encourage open communication
– Encourage all stakeholders and users to point out risks at any time
⚫ Integrate risk management
– Integrate the consideration of risk into the software process

Seven Principles of Risk Management
⚫ Emphasize a continuous process of risk management
– Modify identified risks as more becomes known and add new
risks as better insight is achieved
⚫ Develop a shared product vision
– A shared vision by all stakeholders facilitates better risk
identification and assessment
⚫ Encourage teamwork when managing risk
– Pool the skills and experience of all stakeholders when
conducting risk management activities

Risk Risk type Description
Staff turnover Project Experienced staff will leave the
project before it is finished.
Management change Project There will be a change of
organisational management with
different priorities.
Hardware unavailability Project Hardware which is essential for the
project will not be delivered on
schedule.
Requirements change Project and
product
There will be a larger number of
changes to the requirements than
anticipated.
Specification delays Project and
product
Specifications of essential interfaces
are not available on schedule
Size underestimate Project and
product
The size of the system has been
underestimated.
CASE tool under-
performance
Product CASE tools which support the
project do not perform as anticipated
Technology change Business The underlying technology on which
the system is built is superseded by
new technology.
Product competition Business A competitive product is marketed
before the system is completed.
Software Risks

7
Steps for Risk Management
high probability
high impact
Impact may be
negligible, marginal,
critical, catastrophic
Identify
possible risks
Rank the risks
by probability
and impact
Analyze
each risk
Develop a
contingency
plan

Steps for Risk Management
1)Identify possible risks; recognize what can go wrong
2)Analyze each risk to estimate the probability that it will
occur and the impact (i.e., damage) that it will do if it
does occur
3)Rank the risks by probability and impact
- Impact may be negligible, marginal, critical, and
catastrophic
4)Develop a contingency plan to manage those risks
having high probability and high impact

The Risk Management Process
Risk avoidance
and contingency
plans
Risk planning
Prioritised risk
list
Risk analysis
List of potential
risks
Risk
identification
Risk
assessment
Risk
monitoring

chapter 6.pdf Software Configuration, Quality Assurance and Maintenance

Background
⚫ • Risk identification is a systematic attempt to specify threats to the
project plan
⚫ • By identifying known and predictable risks, the project manager
takes a first step toward avoiding them when possible and controlling
them when necessary
⚫ •Generic risks
⚫ – Risks that are a potential threat to every software project
⚫ •Product-specific risks
⚫ – Risks that can be identified only by those with a clear understanding
of the technology, the people, and the environment that is specific to
the software that is to be built
⚫ This requires examination of the project plan and the statement of scope
⚫ "What special characteristics of this product may threaten our project
plan?"

Risk Item Checklist (Risk Categories)
➢ Product size – risks associated with overall size of the software to be built
➢ Business impact – risks associated with constraints imposed by management or the
marketplace
➢ Customer characteristics – risks associated with sophistication of the customer and
the developer's ability to communicate with the customer in a timely manner
➢ Process definition – risks associated with the degree to which the software process has
been defined and is followed
➢ Development environment – risks associated with availability and quality of the tools
to be used to build the project
➢ Technology to be built – risks associated with complexity of the system to be built and
the "newness" of the technology in the system
➢ Staff size and experience – risks associated with overall technical and project
experience of the software engineers who will do the work

Risk Assessment
⚫ What will change if Risk becomes a problem.

Questionnaire on Project Risk
(Questions are ordered by their relative importance to
project success)
1. Have top software and customer managers formally
committed to support the project?
2. Are end-users enthusiastically committed to the project and
the system/product to be built?
3. Are requirements fully understood by the software
engineering team and their customers?
4. Have customers been involved in the definition of
requirements?
5. Do end-users have realistic expectations?
6. Is project scope stable?

Questionnaire on Project Risk
(Questions are ordered by their relative importance to
project success)
7. Does the software engineering team have the right mix
of skills?
8. Are project requirements stable?
9. Does the project team have experience with the
technology to be implemented?
10. Is the number of people on the project team
adequate to do the job?
11. Do all customer/user constituencies agree on the
importance of the project and on the requirements for
the system/product to be built?

Assessing Overall Project Risk
⚫ If any one of the previous questions is
answered negatively, mitigation,
monitoring and management steps
should be instituted without fail.
⚫ The degree to which the project is at risk
is directly proportional to the number of
negative responses to these questions.

Risk Components and Drivers
The project manager identifies the risk drivers that affect the following risk
components
⚫ Performance risk: the degree of uncertainty that the product will meet
the requirements and be fit for its intended use.
⚫ Support risk: the degree of uncertainty that the resultant software will be
easy to correct, adapt, and enhance.
⚫ Cost risk: the degree of uncertainty that the project budget will be
maintained.
⚫ Schedule risk: the degree of uncertainty that the project schedule will
be maintained and that the product will be delivered on time.
⚫ The impact of each risk driver on the risk component is divided into one of four
impact levels
– Negligible, marginal, critical, and catastrophic

Developed by Reneta Barneva, SUNY Fredonia
Fig 6.1

Background
⚫ impact of risks/likelihood of risk actually
happening
⚫ Risk projection (or estimation) attempts to rate
each risk in two ways
– The probability that the risk is real
– The consequence of the problems associated
with the risk, should it occur

Risk Projection/Estimation steps
⚫ The project planner, along with other managers
and technical staff, performs four risk projection
activities:
– 1. Establish a scale that reflects the perceived likelihood of
a risk(e.g., 1-low, 10-high)
– 2. Delineate the consequences of the risk
– 3. Estimate the impact of the risk on the project and the
product
– 4. Note the overall accuracy of the risk projection so that
there will be no misunderstandings.

Contents of a Risk Table
❑ A risk table provides a project manager with a simple technique for risk
projection
❑ It consists of five columns
▪ Risk Summary – short description of the risk
▪ Risk Category – one of seven risk categories
▪ Probability – estimation of risk occurrence based on group input
▪ Impact – (1) catastrophic (2) critical (3) marginal (4) negligible
▪ RMMM – Pointer to a paragraph in the Risk Mitigation, Monitoring, and
Management Plan

Developing a Risk Table
• List all risks in the first column (by way of the help of the risk item
checklists)
• Mark the category of each risk
• Estimate the probability of each risk occurring
• Assess the impact of each risk based on an averaging of the four risk
components to determine an overall impact value
• Sort the rows by probability and impact in descending order
• Draw a horizontal cutoff line in the table that indicates the risks that will
be given further attention

Risk Projection Cutoff Line
⚫ This line is defined by the project manager
on the risk table.
⚫ Implies that only risks that lie above the line
will be given further attention.
⚫ Risks that fall below the line are re-evaluated
to accomplish second-order prioritization.

Assessing Risk Impact
⚫ Three factors affect the consequences that
are likely if a risk does occur:
– Its nature – problems that are likely if it
occurs
– Its scope – combines the severity with its
overall distribution
– Its timing – when and how long the
impact will be felt

The overall risk exposure formula is
RE = P x C
P = the probability of occurrence for a risk
C = the cost to the project should the risk actually
occur
Example
P = 80% probability that 18 of 60 software
components will have to be developed
C = Total cost of developing 18 components is
$25,000
RE = .80 x $25,000 = $20,000

Assessing Risk Impact
Example Problem
• Risk identification. Only 70 percent of the software components
scheduled for reuse will, in fact, be integrated into the application. The
remaining functionality will have to be custom developed.
• Risk probability. 80% (likely).
• Risk impact. 60 reusable software components were planned. If only
70 percent can be used, 18 components would have to be developed
from scratch (in addition to other custom software that has been
scheduled for development). Since the average component is 100 LOC
and local data indicate that the software engineering cost for each LOC
is $14.00, the overall cost (impact) to develop the components would
be 18 x 100 x 14 = $25,200.
• Risk exposure. RE = 0.80 x 25,200 ~ $20,200.

Risk Assessment
The referent point (break
point):
the point at which the decision
to proceed with the project or
terminate it are equally
weighed.

Risk Mitigation, Monitoring,
and Management
(RMMM)

Risk Mitigation, Monitoring,
and Management

RMMM
• An effective strategy for dealing with risk must consider three issues
(Note: these are not mutually exclusive)
– Risk mitigation (i.e., avoidance)
– Risk monitoring
– Risk management and contingency planning
• Risk mitigation (avoidance) is the primary strategy and is achieved through a
plan
– Example: Risk of high staff turnover
• Risk monitoring
– assessing whether predicted risks occur or not
– ensuring risk aversion steps are being properly applied
– collect information for future risk analysis
– attempt to determine which risks caused which problems
• Risk management
– contingency planning
– actions to be taken in the event that mitigation steps have failed and the risk has
become a live problem

Risk Information Sheets
⚫ Alternative to RMMM in which each risk is
documented individually.
⚫ Often risk information sheets (RIS) are maintained
using a database system.
⚫ RIS components - risk identification, date,
probability, impact, description, refinement,
mitigation/monitoring, management/contingency/
trigger, status, originator, assigned staff member.

Risk: Late Delivery
• Mitigation
The cost associated with a late delivery is critical. A late delivery will result in a late
delivery of a letter of acceptance from the customer. Without the letter of
acceptance, the group will receive a failing grade for the course. Steps have been
taken to ensure a timely delivery by gauging the scope of project based on the
delivery deadline.
• Monitoring
A schedule has been established to monitor project status. Falling behind schedule
would indicate a potential for late delivery. The schedule will be followed closely
during all development stages.
• Management
Late delivery would be a catastrophic failure in the project development. If the
project cannot be delivered on time the development team will not pass the course.
If it becomes apparent that the project will not be completed on time, the only
course of action available would be to request an extension to the deadline form the
customer.

Risk Mitigation Example:
Risk: loss of key team members
• Determine causes of job turnover.
• Eliminate causes before project starts.
• After project starts, assume turnover is going to occur and
work to ensure continuity.
• Make sure teams are organized and distribute information
widely.
• Define documentation standards and be sure documents are
produced in a timely manner.
• Conduct peer review of all work.
• Define backup staff.

Software Quality Assurance
(SQA)

What is Quality Management (SQA)
⚫ Also called software quality assurance (SQA)
⚫ Serves as an umbrella activity that is applied
throughout the software process
⚫ Involves doing the software development
correctly versus doing it over again
⚫ Reduces the amount of rework, which results in
lower costs and improved time to market
56

SQA Encompasses
⚫ Specific quality assurance and quality control tasks (including
formal technical reviews and a multi-tiered testing strategy)
⚫ Effective software engineering practices (methods and tools)
⚫ Control of all software work products and the changes made to
them
⚫ A procedure to ensure compliance with software development
standards
⚫ Measurement and reporting mechanisms

Quality Defined
❑ Defined as a characteristic or attribute of something
❑ Refers to measurable characteristics that we can compare to known standards
❑ Involves such measures as cyclomatic complexity, cohesion, coupling, function
points, and source lines of code
❑ Includes variation control
▪ A software development organization should strive to minimize the variation
between the predicted and the actual values for cost, schedule, and
resources
▪ They should make sure their testing program covers a known percentage of
the software from one release to another
▪ One goal is to ensure that the variance in the number of bugs is also
minimized from one release to another
58

Quality Defined
• Two kinds of quality are sought out
• Quality of design
• The characteristic that designers specify for an item
• This encompasses requirements, specifications, and the design of the
system
• Quality of conformance (i.e., implementation)
• The degree to which the design specifications are followed during
manufacturing
• This focuses on how well the implementation follows the design and
how well the resulting system meets its requirements
⚫ Quality also can be looked at in terms of user satisfaction
User satisfaction = compliant product + good quality + delivery within budget
and schedule

Quality Control
⚫ Involves a series of inspections, reviews, and tests used throughout the
software process
⚫ Ensures that each work product meets the requirements placed on it
⚫ Includes a feedback loop to the process that created the work product
– This is essential in minimizing the errors produced
⚫ Combines measurement and feedback in order to adjust the process
when product specifications are not met
⚫ Requires all work products to have defined, measurable specifications
to which practitioners may compare to the output of each process
60

Quality Assurance Functions
⚫ Consists of a set of auditing and reporting functions that assess the
effectiveness and completeness of quality control activities
⚫ Provides management personnel with data that provides insight into the
quality of the products.
⚫ Alerts management personnel to quality problems so that they can
apply the necessary resources to resolve quality issues
61

The Cost of Quality
⚫ Includes all costs incurred in the pursuit of quality or in performing
quality-related activities
⚫ Is studied to
– Provide a baseline for the current cost of quality
– Identify opportunities for reducing the cost of quality
– Provide a normalized basis of comparison (which is usually dollars)
⚫ Involves various kinds of quality costs (See next slide)
⚫ Increases dramatically as the activities progress from
– Prevention → Detection → Internal failure → External failure
62
"It takes less time to do a thing right than to explain why you did it wrong." Longfellow

Kinds of Quality Costs
❑ Prevention costs
❑Quality planning, formal technical reviews, test equipment,
training
❑ Appraisal costs
❑Inspections, equipment calibration and maintenance, testing
❑ Failure costs – subdivided into internal failure costs and
external failure costs
❑Internal failure costs
❑ Incurred when an error is detected in a product prior to shipment
❑ Include rework, repair, and failure mode analysis
❑External failure costs
❑ Involves defects found after the product has been shipped
❑ Include complaint resolution, product return and replacement, help
line support, and warranty work
63

Software Quality Defined
❑ "Conformance to explicitly stated functional and
performance requirements, explicitly documented
development standards, and implicit characteristics that are
expected of all professionally developed software“
⚫ This definition emphasizes three points
– Software requirements are the foundation from which quality is
measured; lack of conformance to requirements is lack of quality
– Specified standards define a set of development criteria that guide
the manner in which software is engineered; if the criteria are not
followed, lack of quality will almost surely result
– A set of implicit requirements often goes unmentioned; if software
fails to meet implicit requirements, software quality is suspect
64

Software Quality Defined
Software quality is no longer the sole responsibility
of the programmer
❑It extends to software engineers, project
managers, customers, salespeople, and the SQA
group
❑Software engineers apply solid technical methods
and measures, conduct formal technical reviews,
and perform well-planned software testing
65

The SQA Group
⚫ Serves as the customer's in-house representative
⚫ Assists the software team in achieving a high-quality product
⚫ Views the software from the customer's point of view
– Does the software adequately meet quality factors?
– Has software development been conducted according to pre-
established standards?
– Have technical disciplines properly performed their roles as part of
the SQA activity?
⚫ Performs a set of of activities that address quality assurance planning,
oversight, record keeping, analysis, and reporting (See next slide)
66

SQA Activities
⚫ Prepares an SQA plan for a project
⚫ Participates in the development of the project's software process
description
⚫ Reviews software engineering activities to verify compliance with the
defined software process
⚫ Audits designated software work products to verify compliance with those
defined as part of the software process
⚫ Ensures that deviations in software work and work products are
documented and handled according to a documented procedure
⚫ Records any noncompliance and reports to senior management
⚫ Coordinates the control and management of change
⚫ Helps to collect and analyze software metrics
67

Purpose of Reviews
⚫ Serve as a filter for the software process
⚫ Are applied at various points during the software process
⚫ Uncover errors that can then be removed
⚫ Purify the software analysis, design, coding, and testing activities
⚫ Catch large classes of errors that escape the originator more than other
practitioners
⚫ Include the formal technical review (a walkthrough or inspection)
– Acts as the most effective SQA filter
– Conducted by software engineers for software engineers
– Effectively uncovers errors and improves software quality
– Has been shown to be up to 75% effective in uncovering design flaws (which
constitute 50-65% of all errors in software)
⚫ Require the software engineers to expend time and effort, and the organization to
cover the costs
69

Quality control activities
1. Application of technical methods
The use of technical tools and methods helps analyst to achieve high quality
specification and the designer to develop a high quality design
2. Conduct of formal technical reviews (FTR)
Reviews are another quality control activity. Again, reviews are held to find defects
in a work product. The result is changes to the work product. Over time, the
collection of these changes may induce process changes, but that is not
necessary.
It is not the task of team to correct faults, but merely to record them for later
correction.
Defect found early in the software life cycle can be repaired at much less expense
than later in the life cycle.

Families of Review Methods
Method Family Typical Goals Typical Attributes
Walkthroughs Minimal overhead
Developer training
Quick turnaround
Little/no preparation
Informal process
No measurement
Not FTR!
Technical
Reviews
Requirements elicitation
Ambiguity resolution
Training
Formal process
Author presentation
Wide range of discussion
Inspections Detect and remove all
defects efficiently and
effectively
Formal process
Checklists
Measurements
Verify phase

Walkthrough
⚫ a form of software peer review "in which a designer or programmer
leads members of the development team and other interested
parties through a software product, and the participants ask
questions and make comments about possible errors, violation of
development standards, and other problems“
⚫ an interactive process to evoke questions and discussions
time limit - 2 hours SQA, participants: 4 - 6, senior tech. staff
Ways to conduct walkthrough
⚫ Participant-driven: each participant goes through his or her list of
unclear items which may appear incorrect.
⚫ Document-driven: A person responsible for the document walks
the participants through that document, with the reviewers in
everything either with their prepared comments or comments
triggered by the presentation 72

Walkthrough
i. It is not a formal process/review.
ii. It is led by the authors
iii. Author guides the participants through the document according to his or her
thought process to achieve a common understanding and to gather feedback.
iv. Useful for people who are not from the software discipline, who are not used to
or cannot easily understand the software development process.
v. Is especially useful for higher-level documents like requirement specifications,
etc.
The goals of a walkthrough:
i.To present the documents both within and outside the software discipline in order
to gather information regarding the topic under documentation.
ii. To explain or do the knowledge transfer and evaluate the contents of the
document
iii. To achieve a common understanding and to gather feedback.
iv. To examine and discuss the validity of the proposed solutions

Inspection (goes far beyond
walk through)
⚫ An inspection is ‘a visual examination of a software product to detect and identify
software anomalies, including errors and deviations from standards and
specifications
⚫ “Software Inspections are a disciplined engineering practice for detecting and
correcting defects in software artifacts, and preventing their leakage into field
operations.”
Formal steps of inspection
a) an overview of the document is given to the participants
b) participants prepare for inspection, aided with lists of fault types previously found
c) inspection - every piece of logic is covered at least once, and every branch is taken at least once.
A written report is inspection is produce
d) Rework - resolves all faults and problems noted in a written report
e) Follow-up ensures that every single issue raised has been satisfactorily resolved. 74

Formal Technical Review (FTR)
⚫ Objectives
❑ To uncover errors in function, logic, or implementation for any
representation of the software
❑ To verify that the software under review meets its requirements
❑ To ensure that the software has been represented according to predefined
standards
❑ To achieve software that is developed in a uniform manner
❑ To make projects more manageable
⚫ Serves as a training ground for junior software engineers to observe different
approaches to software analysis, design, and construction
⚫ Promotes backup and continuity because a number of people become familiar
with other parts of the software
⚫ May sometimes be a sample-driven review
❑ Project managers must quantify those work products that are the primary
targets for formal technical reviews
❑ The sample of products that are reviewed must be representative of the
products as a whole
75

The FTR Meeting
❑ Has the following constraints
❑From 3-5 people should be involved
❑Advance preparation (i.e., reading) should occur for
each participant but should require no more than
two hours a piece and involve only a small subset of
components
❑The duration of the meeting should be less than two
hours
❑ Focuses on a specific work product (a software
requirements specification, a detailed design, a
source code listing)
76
(More on next slide)

The FTR Meeting
⚫ Activities before the meeting
❑The producer informs the project manager that a work product
is complete and ready for review
❑The project manager contacts a review leader, who evaluates
the product for readiness, generates copies of product
materials, and distributes them to the reviewers for advanced
preparation
❑Each reviewer spends one to two hours reviewing the product
and making notes before the actual review meeting
❑The review leader establishes an agenda for the review
meeting and schedules the time and location

The FTR Meeting (continued)
⚫ Activities during the meeting
❑The meeting is attended by the review leader, all
reviewers, and the producer
❑One of the reviewers also serves as the recorder for all
issues and decisions concerning the product
❑After a brief introduction by the review leader, the
producer proceeds to "walk through" the work product
while reviewers ask questions and raise issues
❑The recorder notes any valid problems or errors that
are discovered; no time or effort is spent in this meeting
to solve any of these problems or errors
78

⚫ Activities following the meeting
❑ The recorder produces a list of review issues that
❑ Identifies problem areas within the product
❑ Serves as an action item checklist to guide the producer in
making corrections
❑ The recorder includes the list in an FTR summary report
❑ This one to two-page report describes what was reviewed, who
reviewed it, and what were the findings and conclusions
❑ The review leader follows up on the findings to ensure that the
producer makes the requested corrections
79

❑ Activities at the conclusion of the meeting
❑All attendees must decide whether to
❑Accept the product without further modification
❑Reject the product due to severe errors (After these
errors are corrected, another review will then occur)
❑Accept the product provisionally (Minor errors need to
be corrected but no additional review is required)
❑All attendees then complete a sign-off in which they
indicate that they took part in the review and that they
concur with the findings

FTR Guidelines
1) Review the product, not the producer
2) Set an agenda and maintain it
3) Limit debate and rebuttal; conduct in-depth discussions off-line
4) Enunciate problem areas but don't attempt to solve the problem noted
5) Take written notes; utilize wallboard to capture comments
6) Limit the number of participants and insist upon advance preparation
7) Develop a checklist for each product in order to structure and focus the review
8) Allocate resources and schedule time for FTRs
9) Conduct meaningful training for all reviewers
10) Review your earlier reviews to improve the overall review process
81

Statistical Software Quality
Assurance

Process Steps
1) Collect and categorize information (i.e., causes)
about software defects that occur
2) Attempt to trace each defect to its underlying
cause (e.g., nonconformance to specifications,
design error, violation of standards, poor
communication with the customer)
3) Using the Pareto principle (80% of defects can
be traced to 20% of all causes), isolate the 20%
83

A Sample of Possible Causes
for Defects
⚫ Incomplete or erroneous specifications
⚫ Misinterpretation of customer communication
⚫ Intentional deviation from specifications
⚫ Violation of programming standards
⚫ Errors in data representation
⚫ Inconsistent component interface
⚫ Errors in design logic
⚫ Incomplete or erroneous testing
⚫ Inaccurate or incomplete documentation
⚫ Errors in programming language translation of design
⚫ Ambiguous or inconsistent human/computer interface
84

McCall’s Quality Factors
• McCall has 11 factors;
• Groups them into categories.
• McCall identified three main perspectives for
characterizing the quality attributes of a software
product.
• These perspectives are:-
o Product revision (ability to change).
o Product transition (adaptability to new environments).
o Product operations (basic operational characteristics).

McCall's software quality
factors

Software Reliability
⚫ Software failure
❑ Defined: Nonconformance to software requirements
❑ Given a set of valid requirements, all software failures can be
traced to design or implementation problems (i.e., nothing wears
out like it does in hardware)
⚫ Software reliability
❑ Defined: The probability of failure-free operation of a software
application in a specified environment for a specified time
❑ Estimated using historical and development data
❑ A simple measure is
MTBF = MTTF + MTTR = Uptime + Downtime
Example:
❑ MTBF = 68 days + 3 days = 71 days
❑ Failures per 100 days = (1/71) * 100 = 1.4
88
Mean time to failure: MTTF
Mean time to repair:MTTR
Mean time between failure: MTBF

SQA Plan: Purpose and Layout
❑ Provides a road map for instituting software quality assurance in an
organization
❑ Developed by the SQA group to serve as a template for SQA activities that
are instituted for each software project in an organization
❑ Structured as follows:
▪ The purpose and scope of the plan
▪ A description of all software engineering work products that fall within
the purview of SQA
▪ All applicable standards and practices that are applied during the
software process
▪ SQA actions and tasks (including reviews and audits) and their
placement throughout the software process
▪ The tools and methods that support SQA actions and tasks
▪ Methods for assembling, safeguarding, and maintaining all SQA-related
records
▪ Organizational roles and responsibilities relative to product quality
89
☺

Software Configuration
Management
⚫ The major danger to a software
configuration is change
– First Law of System Engineering: "No matter
where you are in the system life cycle, the
system will change, and the desire to change
it will persist throughout the life cycle"
10/21/2022 Software Configuration Management (SCM) 90

What is Change Management
⚫ Also called software configuration management (SCM).
⚫ an umbrella activity that is applied throughout the software process
⚫ goal is to maximize productivity by minimizing mistakes caused by confusion
when coordinating software development
⚫ identifies, organizes, and controls modifications to the software being built by
a software development team
⚫ SCM activities are formulated to identify change, control change, ensure that
change is being properly implemented, and report changes to others who
may have an interest
⚫ initiated when the project begins and terminates when the software is taken
out of operation
91

Software Configuration Management (SCM)
➢ Control of the evolution of complex systems
➢ Manages the effects of change throughout the software
process
➢ Identification of individual SCIs & various versions of the
software
➢ Auditing of the software configuration

Management (Contd.)
⚫ The output of the software
process (Software Configuration
Items) are:
i. Computer Programs (both
source level and executable
forms)
ii. Documents that describe the
computer programs (targeted
at both technical practitioners
and users)
iii. Data (contained within the
program or external to it)
Data
Documents
Program
SCIs

Origins of Software Change
⚫ Errors detected in the software need to be corrected
⚫ New business or market conditions dictate changes in product
requirements or business rules
⚫ New customer needs demand modifications of data produced by
information systems, functionality delivered by products, or services
delivered by a computer-based system
⚫ Reorganization or business growth/downsizing causes changes in
project priorities or software engineering team structure
⚫ Budgetary or scheduling constraints cause a redefinition of the system
or product
94

Baseline
⚫ An SCM concept that helps practitioners to control change without
seriously impeding justifiable change
⚫ IEEE Definition: A specification or product that has been formally
reviewed and agreed upon, and that thereafter serves as the basis for
further development, and that can be changed only through formal
change control procedures
⚫ A milestone in the development of software and is marked by the
delivery of one or more computer software configuration items
(CSCIs) that have been approved as a consequence of a formal
technical review
⚫ A CSCI may be such work products as a document, a test suite, or a
software component
95

Baselining Process
1) A series of software engineering tasks produces a CSCI
2) The CSCI is reviewed and possibly approved
3) The approved CSCI is given a new version number and placed
in a project database (i.e., software repository)
4) A copy of the CSCI is taken from the project database and
examined/modified by a software engineer
5) The baselining of the modified CSCI goes back to Step #2
96

▪Configuration Item (CI)
refers to the
fundamental structural
unit of a SCM
▪Deliverables of Large
Software Development
Effort
Software Configuration Items
(SCIs)
Deliverables
SRS
Design
Documents
Test Cases
Source Code
User Manual

Items (SCIs)
10/21/2022 98

Items (SCIs)
• In reality, SCIs are
organized to form
configuration objects
that may be cataloged
in the project database
with a single name.
• A configuration object
has a name, attributes,
and is “connected” to
other objects by
relationships.

Automated SCM Repository
(Functions and Tools)
100
Functions
Data integrity
Information sharing
Tool integration
Data integration
Methodology enforcement
Document standardization
Versioning
Dependency
tracking
Change
management
Requirements
tracing
Configuration
management
Audit
trails
SCM Repository
(Explained on next two slides)

Functions of an SCM
Repository
❑ Data integrity
Validates entries, ensures consistency, cascades modifications
❑ Information sharing
Shares information among developers and tools, manages and controls multi-user
access
❑ Tool integration
Establishes a data model that can be accessed by many software engineering
tools, controls access to the data
❑ Data integration
Allows various SCM tasks to be performed on one or more CSCIs
❑ Methodology enforcement
Defines an entity-relationship model for the repository that implies a specific
process model for software engineering
❑ Document standardization
Defines objects in the repository to guarantee a standard approach for creation of
software engineering documents 101

Toolset Used on a
Repository
❑ Versioning
❑ Save and retrieve all repository objects based on version number
❑ Dependency tracking and change management
❑ Track and respond to the changes in the state and relationship of all objects in the
repository
❑ Requirements tracing
❑ (Forward tracing) Track the design and construction components and deliverables that
result from a specific requirements specification
❑ (Backward tracing) Identify which requirement generated any given work product
❑ Configuration management
❑ Track a series of configurations representing specific project milestones or production
releases
❑ Audit trails
❑ Establish information about when, why, and by whom changes are made in the repository
102

SCM Process
Primary Objectives:
1. To identify all items that
collectively define the
software configuration
2. To manage changes to
one or more of these
items
3. To facilitate the
construction of different
versions of an
application
4. To ensure that software
quality is maintained as
the configuration evolves
over time

SCM Tasks (continued)
❑ Concentric layers (from inner to outer)
❑ Identification
❑ Change control
❑ Version control
❑ Configuration auditing
❑ Status reporting
❑ CSCIs flow outward through these layers during their life cycle
❑ CSCIs ultimately become part of the configuration of one or
more versions of a software application or system
104

Identification of Objects
⚫ To control & manage SCIs, each should be separately
named & then organized using an object-oriented
approach.
⚫ Types of objects:
i. Basic objects
⚫ Unit of information that is created during analysis, design, code or test.
⚫ For Example: Part of design model, source code for a component,
suite of test cases, etc.
ii. Aggregate objects
⚫ Collection of basic objects & other another objects.
⚫ For Example: Design Specification
10/21/2022 105

Identification Task
❑ Each object has a set of distinct features that identify it
▪ A name that is unambiguous to all other objects
▪ A description that contains the CSCI type, a project
identifier, and change and/or version information
▪ List of resources needed by the object
▪ The object realization (i.e., the document, the file, the
model, etc.)

Change Control
10/21/2022 107
➢Procedural activity that ensures quality & consistency as changes
are made to a configuration object.
➢Begins with a change request, leading to a decision to make or
reject the request for change.
➢ A change request is submitted to a configuration control
authority, which is usually a change control board (CCB)
➢ The request is evaluated for technical merit, potential side effects, the overall
impact on other configuration objects and system functions, and projected cost in
terms of money, time, and resources

Change Control
➢ An engineering change order (ECO) is issued for each
approved change request
➢ Describes the change to be made, the constraints to follow, and the criteria
for review and audit
➢ The baselined CSCI is obtained from the SCM repository
➢ Access control governs which software engineers have the authority to
access and modify a particular configuration object
➢ Synchronization control helps to ensure that parallel changes performed
by two different people don't overwrite one another

Version Control
➢ Combines procedures & tools to manage versions of
configuration objects that are created during the software
process
➢ A new version is defined when major changes have been made
to one or more objects

Version Control Task
❑ Version control is a set of procedures and tools for managing
the creation and use of multiple occurrences of objects in the
SCM repository
❑ Required version control capabilities
▪ An SCM repository that stores all relevant configuration objects
▪ A version management capability that stores all versions of a
configuration object (or enables any version to be constructed using
differences from past versions)
▪ A make facility that enables the software engineer to collect all relevant
configuration objects and construct a specific version of the software
▪ Issues tracking (bug tracking) capability that enables the team to record
and track the status of all outstanding issues associated with each
configuration object
112

Version Control Task
❑ The SCM repository maintains a change set
▪ Serves as a collection of all changes made to a baseline
configuration
▪ Used to create a specific version of the software
▪ Captures all changes to all files in the configuration along with the
reason for changes and details of who made the changes and
when
113

Configuration Audit
➢ To ensure that change has been properly implemented:
i. Formal Technical Reviews
ii. Software Configuration Audit.
➢ Formal Technical Reviews
➢ Software Quality Assurance (SQA) activity performed by
software engineers (and others)
➢ FTR serves as a training ground, enabling junior engineers to
observe different approaches to software analysis, design, and
implementation
➢ Software Configuration Audit
➢ SQA Activity
➢ Helps to ensure that quality is maintained as changes are
made

Formal Technical Reviews & Configuration
Audit
❑ Formal technical review addresses the following questions
▪ Has the change specified in the ECO been made? Have any additional
modifications been incorporated?
▪ Has a formal technical review been conducted to assess technical correctness?
▪ Has the software process been followed, and have software engineering
standards been properly applied?
▪ Has the change been "highlighted" and "documented" in the CSCI? Have the
change data and change author been specified? Do the attributes of the
configuration object reflect the change?
▪ Have SCM procedures for noting the change, recording it, and reporting it been
followed?
▪ Have all related CSCIs been properly updated?
❑ A configuration audit ensures that
▪ The correct CSCIs (by version) have been incorporated into a specific build
▪ That all documentation is up-to-date and consistent with the version that has
been built
115

Status Reporting
⚫ Configuration Status
Reporting (Status
Accounting) is an
SCM task that
answers the
following questions:
i. What happened?
ii. When did it
happen?
iii. Who did it?
iv. What else will be
affected?

Status Reporting Task
⚫ Sources of entries for configuration status
reporting
– Each time a CSCI is assigned new or updated
information
– Each time a change is approved by the CCB and an
ECO is issued
– Each time a configuration audit is conducted
⚫ The configuration status report
– Placed in an on-line database or on a website for
software developers and maintainers to read
– Given to management and practitioners to keep them
appraised of important changes to the project CSCIs
117

chapter 6.pdf Software Configuration, Quality Assurance and Maintenance

More Related Content

Similar to chapter 6.pdf Software Configuration, Quality Assurance and Maintenance (20)

Recently uploaded (20)

chapter 6.pdf Software Configuration, Quality Assurance and Maintenance