Requirements Validation

Validation Techniques Assistance Techniques for Validation Summary

Requirements Validation

Antonio Villegas Ni˜o
n
[email protected]
@avillegasn

Requirements Engineering
Facultat d’Inform`tica de Barcelona
a

April 4, 2011

C
CC BY: $

Antonio Villegas Requirements Validation April 4, 2011 1 / 29

Validation Techniques Assistance Techniques for Validation Summary

Outline

1 Fundamentals

2 Validation Techniques

3 Assistance Techniques for Validation

Antonio Villegas Requirements Validation April 4, 2011 2 / 29

Validation Techniques Assistance Techniques for Validation Summary

The context of Requirements Validation

Antonio Villegas Requirements Validation April 4, 2011 3 / 29

Validation Techniques Assistance Techniques for Validation Summary

The context of Requirements Validation

Antonio Villegas Requirements Validation April 4, 2011 3 / 29

Validation Techniques Assistance Techniques for Validation Summary

Motivation and Goals

Requirements Validation

Validation (in requirements engineering)

Validation denotes checking whether inputs, performed activities,
and created outputs (requirements artifacts) of the requirements
engineering core activities fulfil defined quality criteria.

Validation is performed by involving relevant stakeholders, other
requirement sources (standards, laws, etc.) as well as external
reviewers, if necessary.

Antonio Villegas Requirements Validation April 4, 2011 4 / 29

Validation Techniques Assistance Techniques for Validation Summary

Motivation and Goals

Quality Gateway

Antonio Villegas Requirements Validation April 4, 2011 5 / 29

Validation Techniques Assistance Techniques for Validation Summary

Motivation and Goals

Quality Criteria

Completeness
– The requirement must contain all relevant information
(template).
Consistency
– The requirements must be compatible with each other.
Adequacy
– The requirements must address the actual needs of the system.
Unambiguity
– Every requirement must be described in a way that precludes
different interpretations.
Comprehensibility
– The requirements must be understandable by the stakeholders.

Antonio Villegas Requirements Validation April 4, 2011 6 / 29

Validation Techniques Assistance Techniques for Validation Summary

Motivation and Goals

Quality Criteria
Importance
– Each requirement must indicate how essential it is for the
success of the project.
Measurability
– The requirement must be formulated at a level of precision
that enables to evaluate its satisfaction.
Necessity
– The requirements must all contribute to the satisfaction of the
project goals.
Viability
– All requirements can be implemented with the available
technology, human resources and budget.
Traceability
– The context in which a requirement was created should be
easy to retrieve.
Antonio Villegas Requirements Validation April 4, 2011 7 / 29

Validation Techniques Assistance Techniques for Validation Summary

Motivation and Goals

Examples

Train doors shall be opened
as soon as the train is stopped at a platform

Antonio Villegas Requirements Validation April 4, 2011 8 / 29

Validation Techniques Assistance Techniques for Validation Summary

Motivation and Goals

Examples

Ambiguity
Train doors shall be opened
as soon as the train is stopped at a platform

Possible interpretations:
The front of the train is (stopped) at a platform?
or
The whole train is (stopped) at a platform?

Antonio Villegas Requirements Validation April 4, 2011 8 / 29

Validation Techniques Assistance Techniques for Validation Summary

Motivation and Goals

Examples

Contradiction
Train doors must always be kept closed between stations.

and elsewhere…

Train doors must be opened once a train is stopped after an
emergency signal.

Antonio Villegas Requirements Validation April 4, 2011 9 / 29

Validation Techniques Assistance Techniques for Validation Summary

Motivation and Goals

Risk of insufficient validation

Quality Assurance

Antonio Villegas Requirements Validation April 4, 2011 10 / 29

Validation Techniques Assistance Techniques for Validation Summary

Motivation and Goals

Risk of insufficient validation

Antonio Villegas Requirements Validation April 4, 2011 11 / 29

Validation Techniques Assistance Techniques for Validation Summary

Motivation and Goals

Risk of insufficient validation

The earlier an error is discovered,
the cheaper it is to correct.

Antonio Villegas Requirements Validation April 4, 2011 12 / 29

Validation Techniques Assistance Techniques for Validation Summary

Principles of Validation

The 6 Principles of Validation

First Principle: Involving the Right Stakeholders
Ensure that relevant company-internal as well as relevant external
stakeholders participate in validation.

Pay attention to the reviewers’ independence and appoint external,
independent stakeholders, if necessary.

Second Principle: Defect Detection vs. Defect Correction
Separate defect detection from the correction of the detected
defects.

Antonio Villegas Requirements Validation April 4, 2011 13 / 29

Validation Techniques Assistance Techniques for Validation Summary

Principles of Validation

The 6 Principles of Validation

First Principle: Involving the Right Stakeholders
Ensure that relevant company-internal as well as relevant external
stakeholders participate in validation.

Pay attention to the reviewers’ independence and appoint external,
independent stakeholders, if necessary.

Second Principle: Defect Detection vs. Defect Correction
Separate defect detection from the correction of the detected
defects.

Antonio Villegas Requirements Validation April 4, 2011 13 / 29

Validation Techniques Assistance Techniques for Validation Summary

Principles of Validation

The 6 Principles of Validation

Third Principle: Leveraging Multiple Independent Views
Whenever possible, try to obtain independent views that can be
integrated during requirements validation in order to detect defects
more reliably.

Fourth Principle: Use of Appropriate Documentation Formats
Consider changing the documentation format of the requirements
into a format that matches the validation goal and the preferences
of the stakeholders who actually perform the validation.

Antonio Villegas Requirements Validation April 4, 2011 14 / 29

Validation Techniques Assistance Techniques for Validation Summary

Principles of Validation

The 6 Principles of Validation

Third Principle: Leveraging Multiple Independent Views
Whenever possible, try to obtain independent views that can be
integrated during requirements validation in order to detect defects
more reliably.

Fourth Principle: Use of Appropriate Documentation Formats
Consider changing the documentation format of the requirements
into a format that matches the validation goal and the preferences
of the stakeholders who actually perform the validation.

Antonio Villegas Requirements Validation April 4, 2011 14 / 29

Validation Techniques Assistance Techniques for Validation Summary

Principles of Validation

The 6 Principles of Validation

Fifth Principle: Creation of Development Artefacts during
Validation
If your validation approach generates poor results, try to support
defect detection by creating development artefacts such as
architectural artefacts, test artefacts, user manuals, or goals and
scenarios during validation.

Sixth Principle: Repeated Validation
Establish guidelines that clearly determine when or under what
conditions an already released requirements artefact has to be
validated again.

Antonio Villegas Requirements Validation April 4, 2011 15 / 29

Validation Techniques Assistance Techniques for Validation Summary

Principles of Validation

The 6 Principles of Validation

Fifth Principle: Creation of Development Artefacts during
Validation
If your validation approach generates poor results, try to support
defect detection by creating development artefacts such as
architectural artefacts, test artefacts, user manuals, or goals and
scenarios during validation.

Sixth Principle: Repeated Validation
Establish guidelines that clearly determine when or under what
conditions an already released requirements artefact has to be
validated again.

Antonio Villegas Requirements Validation April 4, 2011 15 / 29

Validation Techniques Assistance Techniques for Validation Summary

Validation Techniques

Inspections
Desk-Checks
Walkthroughs
Prototypes

Antonio Villegas Requirements Validation April 4, 2011 16 / 29

Validation Techniques Assistance Techniques for Validation Summary

Inspections

Inspections

Inspection: an organized examination process of the requirements.

Involved roles: Critical Success Factors:
Organizer Commitment of the organization
Moderator Size and complexity of the
inspected artefacts
Author
Number and experience of the
Inspectors
inspectors
Minute-taker
Benefit: Detailed
checking of the artefacts Effort: Medium-High

Antonio Villegas Requirements Validation April 4, 2011 17 / 29

Validation Techniques Assistance Techniques for Validation Summary

Desk-Checks

Desk-Checks

1 The author of a requirement artefact distributes the artefact to a
set of stakeholders.
2 The stakeholders check the artefact individually.
3 The stakeholders report the identified defects to the author.
4 The collected issues are discussed in a group session (optional).

Critical Success Factors: Benefit: Obtain feedback
Commitment of the participants from individual reviewers
Coverage of all the aspects
Not recommended for critical
artefacts Effort: Medium

Antonio Villegas Requirements Validation April 4, 2011 18 / 29

Validation Techniques Assistance Techniques for Validation Summary

Walkthroughs

Walkthroughs

A walkthrough does not have formally defined procedure and does
not require a differentiated role assignment.
Checking early whether an idea is feasible or not.
Obtaining the opinion and suggestions of other people.
Checking the approval of others and reaching agreement.

Critical Success Factors: Benefit: Validation of
Involving stakeholders from ideas and sketches
different contexts
Comprehensible presentation of the
artefact Effort: Medium-Low

Antonio Villegas Requirements Validation April 4, 2011 19 / 29

Validation Techniques Assistance Techniques for Validation Summary

Prototypes

Prototypes

A prototype allows the stakeholders to try out the requirements for
the system and experience them thereby.
1 Develop the prototype (tool support).
2 Training of the stakeholders.
3 Observation of prototype usage.
4 Collect issues.

Critical Success Factors: Benefit:
Effort Highly effective defect detection
Level of detail of the Proof of feasibility
prototype
Quality of the review Effort: Very High-High

Antonio Villegas Requirements Validation April 4, 2011 20 / 29

Validation Techniques Assistance Techniques for Validation Summary

Review

Review of Techniques

Inspection Desk-Check Walkthrough Prototype
Goal extensive search for less-detailed check feedback on early try out the
defects in a of a larger extent sketches of an requirements and
manageable extent of artefacts artefact experience them
of requirements individually
artefacts
Group session yes, strict focus on optional yes, for providing yes, to evaluate the
defect collection feedback observation
Process detailed roles and defined but flexible no predefined defined but flexible
strict steps procedure
Effort for Medium-High Medium Medium-Low Very High-High
execution
Benefit for High Medium Low Very High
validation

Antonio Villegas Requirements Validation April 4, 2011 21 / 29

Validation Techniques Assistance Techniques for Validation Summary

Assistance Techniques for Validation

Checklists
Perspective-based
Reading
Creation of Artefacts

Antonio Villegas Requirements Validation April 4, 2011 22 / 29

Validation Techniques Assistance Techniques for Validation Summary

Checklists

Checklists

A checklist should comprise few questions so that the reviewer can
use it during validation in an effective way.
Applicable with each validation technique.
Create different checklists for different validation goals
Create checklists based on rules, experience, and defect statistics.

Critical Success Factors: Benefit:
Limiting the size of the checklists Each reviewer knows how
to check the artefact
Avoiding generic questions such as
”Are the requirements complete?” Used for validation and for
elicitation
Considering the stakeholders’
background and experience Effort: Low-Very Low

Antonio Villegas Requirements Validation April 4, 2011 23 / 29

Validation Techniques Assistance Techniques for Validation Summary

Perspective-based Reading

Perspective-based Reading
Applying perspective-based reading during requirements validation
means checking the same set of artefacts by reading them from
different perspectives.
Applicable with each validation technique.
Identify the perspectives that are relevant for the project
For each perspective, provide reading instructions and questions
that support defect detection.

Critical Success Factors: Benefit:
Quality of the instructions and Practical support for
questions reviewers
Experience of the reviewers More defects uncovered
than without perspectives
Involving stakeholders from
different contexts Effort: High-Medium
Antonio Villegas Requirements Validation April 4, 2011 24 / 29

Validation Techniques Assistance Techniques for Validation Summary

Creation of Artifacts

Creation of Artifacts

Creating artefacts during validation means using the requirements
to be checked as a reference for creating other development
artefacts.
Applicable with each validation technique.
Verbalisation of Models.
Creating Scenarios, Test Cases, a User Manual…

Benefit:
Critical Success Factors:
Defects uncovered during
Quality of the created artefacts artefacts creation
Experience of the reviewers Reuse of the created
Traceability between requirements artefacts as initial drafts
and artefacts
Effort: High-Medium

Antonio Villegas Requirements Validation April 4, 2011 25 / 29

Validation Techniques Assistance Techniques for Validation Summary

Review

Review of Assistance Techniques

Checklists Perspective-based Creation of
Reading Artifacts
Goal guide the checking the same validate the
stakeholders during set of artefacts by requirements while
the requirements reading them from creating drafts of
validation different artefacts
perspectives
Critical Success size and diversity of quality of the quality and
Factors the checklists instructions and traceability of the
experience of the artefacts
reviewers
Effort for execution Low-Very Low High-Medium High-Medium
Benefit for validation Medium Medium High-Medium

Antonio Villegas Requirements Validation April 4, 2011 26 / 29

Validation Techniques Assistance Techniques for Validation Summary

Summary

The Context of Requirements Validation in RE

The Quality Gateway and Quality Criteria

Risk of Insufficient Validation

The 6 Principles of Validation

Validation Techniques

Assistance Techniques for Validation

Antonio Villegas Requirements Validation April 4, 2011 27 / 29

Validation Techniques Assistance Techniques for Validation Summary

“Quality is never an accident.
It is always the result of intelligent effort.”

John Ruskin

Antonio Villegas Requirements Validation April 4, 2011 28 / 29

Validation Techniques Assistance Techniques for Validation Summary

References

Denger, C. and Olsson, T. Quality Assurance in Requirements Engineering

Pohl, K. Requirements Engineering: Fundamentals, Principles, and
Techniques.
Part V. Validation pp. 509-585

van Lamsweerde, A. Requirements Engineering: from system goals to UML
models to software specifications.
Chapter 5. Requirements Quality Assurance pp. 187-217

Robertson, S. and Robertson, J. Mastering the Requirements Process.
2nd Edition. Chapter 11.

Antonio Villegas Requirements Validation April 4, 2011 29 / 29