Quality Attribute Workshop(QAW) Method - The first knowledge sharing application in Vietnam

QAW Method

The QAW is a facilitated, early intervention method used to generate,
prioritize, and refine quality attribute scenarios before the software
architecture is completed. The QAW is focused on system-level concerns and
specifically the role that software will play in the system. The QAW is
dependent on the participation of system stakeholders—individuals on whom the
system has significant impact, such as end users, installers, administrators
(of database management systems [DBMS], networks, help desks, etc.), trainers,
architects, acquirers, system and software engineers, and others. The group of
stakeholders present during any one QAW should number at least 5 and no more
than 30 for a single workshop. In preparation for the workshop, stakeholders
receive a “participants handbook” providing example quality attribute
taxonomies, questions, and scenarios. If time allows, the handbook should be
customized to the domain of the system and contain the quality attributes,
questions, and scenarios that are appropriate to the domain and the level of
architectural detail available.

The contribution of each stakeholder is essential during a QAW; all
participants are expected to be fully engaged and present throughout the
workshop. Participants are encouraged to comment and ask questions at any time
during the workshop. However, it is important to recognize that facilitators
may occasionally have to cut discussions short in the interest of time or when
it is clear that the discussion is not focused on the required QAW outcomes.
The QAW is an intense and demanding activity. It is very important that all
participants stay focused, are on time, and limit side discussions throughout
the day.

The QAW
involves the following steps:

1.QAW
Presentation and Introductions

2.Business/Mission
Presentation

3.Architectural
Plan Presentation

4.Identification
of Architectural Drivers

5.Scenario
Brainstorming

6.Scenario
Consolidation

7.Scenario
Prioritization

8.Scenario
Refinement

The following sections describe each step of the QAW in detail.

Step 1: QAW Presentation and Introductions

In this step, QAW facilitators describe the motivation for the QAW and
explain each step of the method. We recommend using a standard slide
presentation that can be customized depending on the needs of the sponsor. Next,
the facilitators introduce themselves and the stakeholders do likewise, briefly
stating their background, their role in the organization, and their
relationship to the system being built.

Step 2: Business/Mission Presentation

After Step 1, a representative of the stakeholder community presents the
business and/or mission drivers for the system. The term “business and/or
mission drivers” is used carefully here.

Some organizations are clearly motivated by business concerns such as profitability,
while others, such as governmental
organizations, are motivated by mission concerns and find profitability
meaningless. The stakeholder representing the business and/or mission concerns
(typically a manager or management
representative) spends about one hour presenting the system’s
business/mission context

• high-level functional requirements, constraints, and quality attribute
requirements

During the presentation, the facilitators listen carefully and capture
any relevant information that may shed light on the quality attribute drivers.
The quality attributes that will be refined in later steps will be derived
largely from the business/mission needs presented in this step.

Step 3: Architectural Plan Presentation

While a detailed system architecture might not exist, it is possible
that high-level system descriptions, context drawings, or other artifacts have
been created that describe some of the system’s technical details. At this
point in the workshop, a technical stakeholder will present the system
architectural plans as they stand with respect to these early documents.
Information in this presentation may include

• plans and strategies for how key business/mission requirements will be
satisfied

• key technical requirements and constraints—such as mandated operating
systems, hardware, middleware, and standards—that will drive architectural
decisions

• presentation of existing context diagrams, high-level system diagrams,
and other written Descriptions

Step 4: Identification of Architectural Drivers

During
steps 2 and 3, the facilitators capture information regarding architectural
drivers that are key to realizing quality attribute goals in the system. These
drivers often include high-level requirements, business/mission concerns, goals
and objectives, and various quality attributes. Before undertaking this step,
the facilitators should excuse the group for a 15-minute break, during which
they will caucus to compare and consolidate notes taken during steps 2 and 3.
When the stakeholders reconvene, the facilitators will share their list of key
architectural drivers and ask the stakeholders for clarifications, additions,
deletions, and corrections. The idea is to reach a consensus on a distilled
list of architectural drivers that include high-level requirements, business
drivers, constraints, and quality attributes. The final list of architectural
drivers will help focus the stakeholders during scenario brainstorming to
ensure that these concerns are represented by the scenarios collected.

Step 4: Identification of Architectural Drivers

Step 5: Scenario Brainstorming

After the
architectural drivers have been identified, the facilitators initiate the
brainstorming process in which stakeholders generate scenarios. The
facilitators review the parts of a good scenario (stimulus, environment, and
response) and ensure that each scenario is well formed during the workshop.

Each
stakeholder expresses a scenario representing his or her concerns with respect
to the system in round-robin fashion. During a nominal QAW, at least two
round-robin passes are made so that each stakeholder can contribute at least
two scenarios. The facilitators ensure that at least one representative
scenario exists for each architectural driver listed in Step 4. Scenario
generation is a key step in the QAW method and must be carried out with care.
We suggest the following guidance to help QAW facilitators during this step:

1.
Facilitators should help stakeholders create well-formed scenarios. It is
tempting for stakeholders to recite requirements such as “The system shall
produce reports for users.”

While
this is an important requirement, facilitators need to ensure that the quality
attribute aspects of this requirement are explored further. For example, the
following scenario sheds more light on the performance aspect of this
requirement: “A remote user requests a
database report via the Web during
peak usage and receives the report within five seconds. ”

Note that
the initial requirement hasn’t been lost, but the scenario further explores the
performance aspect of this requirement. Facilitators should note that quality
attribute names by themselves are not enough. Rather than say “the system shall be modifiable,” the
scenario should describe what it means to be modifiable by providing a specific
example of a modification to the system vis-à-vis a scenario.

2. The
vocabulary used to describe quality attributes varies widely. Heated debates
often revolve around to which quality attribute a particular system property
belongs. It doesn’t matter what we call a particular quality attribute, as long
as there’s a scenario that describes what it means.

3.Facilitators
need to remember that there are three general types of scenarios and to ensure
that each type is covered during the QAW:

a. use case
scenarios – involving anticipated uses of the system b. growth scenarios –
involving anticipated changes to the system

c.
exploratory scenarios – involving unanticipated stresses to the system that can
include uses and/or changes

4.Facilitators
should refer to the list of architectural drivers generated in Step 4 from time
to time during scenario brainstorming to ensure that representative scenarios
exist for each one.

Step 6: Scenario Consolidation

After the
scenario brainstorming, similar scenarios are consolidated when reasonable.

To do
that, facilitators ask stakeholders to identify those scenarios that are very
similar in content. Scenarios that are similar are merged, as long as the
people who proposed them agree and feels that their scenarios will not be
diluted in the process. Consolidation is an important step because it helps to
prevent a “dilution” of votes during the prioritization of scenarios (Step 7).

Such a dilution occurs when stakeholders split
their votes between two very similar scenarios. As a result, neither scenario
rises to importance and is therefore never refined (Step 8). However, if the
two scenarios are similar enough to be merged into one, the votes might be
concentrated, and the merged scenario may then rise to the appropriate level of
importance and be refined further.

Facilitators
should make every attempt to reach a majority consensus with the stakeholders
before merging scenarios. Though stakeholders may be tempted to merge scenarios
with abandon, they should not do so. In actuality, very few scenarios are
merged.

Step 7: Scenario Prioritization

Prioritization
of the scenarios is accomplished by allocating each stakeholder a number of
votes equal to 30% of the total number of scenarios generated after
consolidation. The actual number of votes allocated to stakeholders is rounded
to an even number of votes at the discretion of the facilitators. For example,
if 30 scenarios were generated, each stakeholder gets 30 x

0.3, or
9, votes rounded up to 10. Voting is done in round-robin fashion, in two
passes. During each pass, stakeholders allocate half of their votes.
Stakeholders can allocate any number of their votes to any scenario or
combination of scenarios. The votes are counted, and the scenarios are
prioritized accordingly.

Step 8: Scenario Refinement

After the
prioritization, depending on the amount of time remaining, the top four or five
scenarios are refined in more detail. Facilitators further elaborate each one,
documenting the following:

• Further
clarify the scenario by clearly describing the following six things:

1.stimulus
– the condition that affects the system

2.response
– the activity that results from the stimulus

3.source of
stimulus – the entity that generated the stimulus

4.environment
– the condition under which the stimulus occurred

5.artifact
stimulated – the artifact that was stimulated

6.response
measure – the measure by which the system’s response will be evaluated

•
Describe the business/mission goals that are affected by the scenario.

• Describe
the relevant quality attributes associated with the scenario.

• Allow the
stakeholders to pose questions and raise any issues regarding the scenario.
Such questions should concentrate on the quality attribute aspects of the
scenario and any concerns that the stakeholders might have in achieving the
response called for in the scenario.

See the
example template for scenario refinement in Appendix A. This step continues
until time runs out or the highest priority scenarios have been refined.
Typically, time runs out first.