Introduction to Software Process Models

1. Introduction

Process, and Process Models are considered, by many people, to be
among the most theoretical areas of software engineering. This
view reminds me of the response an Irish musician once gave
an American interviewer who asked if there was any Irish tradition
of fight songs. His answer was:

Is there, in fact, any value in “singin’ about it first”?
Why should we study and discuss abstract models of development processes?
It turns out there are some very practical benefits to this study:

It gives us a richer vocabulary of terms for discussing
software development and the ways we propose to organize
our work.
Development models are simple ways of organizing
development plans, and effective metaphors for describing
them to others.
Different models are not merely abstractions. They
are suited to different types of problems.
Choosing the right model sets reasonable expectations,
and lays a foundation for an effective attack on the
problem.
Choosing the wrong model sets false expectations, and
sketches out a plan for failure.

2. Software Development Activities

In the readings on formal process models, and agile process we will
see that there are a great many ways to structure the software
development process. All models, however, seem to agree that the
work can be divided into a few basic phases.

definition … figuring out what to build.This includes activities like creating the initial concept,
developing product descriptions, gathering requirements,
and negotiating specifications.
planning … figuring out how to build it.This includes both technical planning (architecture, designs,
modeling, prototypes) and managerial planning (people, budgets,
schedules) and the validation of those plans.
construction … building it.This includes the coding of both the product and testing
tools, unit testing, the development of content and
documentation, integration, whole system testing, and
acceptance testing.
deployment … getting it to its users.This includes activities like release packaging and product
manufacturing, early access programs, distribution, support,
bug-fixing, patches, and on-going maintenance and enhancement.

Exactly what each one of these activities implies may be different
for every project, but most of these activities are (in some form)
applicable to most software projects. Where people differ is on how
to structure these phases and activities into a software project.

3. Software Process Models

There are a few basic models, into which the above phases are
traditionally organized.

3.1 The Waterfall Model

The first, and most primitive development model is the
waterfall model so named because its diagram
resembles a succession of cascading waterfalls, where each
waterfall in the cascade feeds the pond that supplies the next.
It is a simple and intuitive model, where each phase is completed (and verified)
before the next phase is begun, and the whole sequence culminates
in a finished product.

Experience has shown this model to be a simpler solution than most
interesting software projects will admit of.
We seldom know enough (about the problem to be solved or
the obstacles we will encounter) to enable us to completely specify
the solution up front. As a result, very few problems can actually
be solved in a simple one-pass waterfall project.

If we cannot solve our problem in one-pass, we will have to solve
it with a series of successive approximations. All of the other models
represent different ways of structuring that succession.

3.2 The Incremental Model

Often, we have a grand vision, but we clearly understand how to realize
only a small subset of it. If that initial subset would be a useful
product in its own right, there is no reason for us to even try to
create the whole grand vision in a single project.

Define, plan, build, and deploy the initial subset that we understand.
In this process we will learn valuable lessons about both the problems
to be solved, and how users use the software. Based on this new information,
we can define and plan the next release, with additional features.

This model of successive releases is employed in most major software
projects. Each successive release is much simpler than the grand vision,
and we only build the parts for which we have a good understanding of
the requirements and design. This approach has the potential to get
us to our desired end point, in a series of smaller, simpler, safer steps.

3.3 Iteratitive vs. Incremental (Research vs. Development)

It is important to recognize the difference between research and development:

a development model is appropriate when we are
pretty sure that we know what we want to build, and how to build it.
A development project is expected to culminate in a product..
If we are trying to figure out what we should build, or how to
build it, those are research problems.
A research project is expected to culminate in answers to questions,
and usually additional questions.

The difference between Incremental and Iterative models is the difference
between a cascading succession of development projects, and a cascading
succession of research projects.

If our goal is to figure out what to build (or how to build it), we cannot
be certain that, at the end of the next cycle of activities, we will have
a product, or that we will even have a clear definition and plan for a product.
We may have go through many iterations of definition, planning, construction,
and evaluation before we are prepared to define and plan a product cycle.
Because these models go through an unpredictable succession of development
cycles, they
are often referred to as spiral models).

3.4 Choosing a Model

The decision of whether to use a waterfall, incremental, or iterative model
is not a matter of personal choice. It is a characterization of our goals
and how confident we are of them:

If we are building a one-time project with clear requirements,
and no major problems anticipated, a simple waterfall model
is the right one.
If we are planning to deliver a succession of products, each
with incremental capabilities, and benefiting from lessons
learned in prior releases, an incremental model describes
our approach.
If we are investigating potential products or ways of building
them, and our goal is to experiment with and validate
alternative approaches, a spiral model is the one that best
suits the problem.

You are not forced to choose one model or the other. You are free
to arrange activities and to create a hybrid model that best fits your
problem.
For instance, it might seem tautological that you can’t (or at least shouldn’t)
build something until after you have figured out what to build.
But projects often involve building many different things, some
of which are much better understood than others. Must we have
defined everything before we can start building
anything?

If we started building a well-understood task A, and
then later found that the best solution to task B
involved a redefinition of task A, we might find
ourselves having wasted a great deal of effort.
If we were sure that task A was truly independent
of task B, but the same people would be used for
clarifying task B and implementing task A, doing
task A first
might be accelerating a low-risk activity at the
cost of delaying a higher-risk activity. It is
generally prudent to deal with higher risk problems
sooner.
But if task A was truly independent of task B
(in both requirements and resources) applying our
developers and testers to task A while the managers
and architects argued about task B might be a very
good use of both time and resources.

How safe it is to overlap project phases depends on how confident
we are that we have correctly understood the work that we propose
to start sooner. The benefits from such phase overlap depend on
whether or not there are different resources that can reasonably
be working on the different phases in parallel.

4. Processes Definitions

The high level models describe a general sequencing of activities,
and set expectations for what kind of a project we are undertaking.
They do not actually tell us what we have to do. In section 2,
I gave a sample list of activities for each phase, but for any
given project, this list might have inappropriate or be missing
important activities. Any one of those activities (e.g. gathering
requirements) could be broken down into a more detailed list of
tasks:

identifying potential stake holders
preliminary requirements elicitation
clarification of unclear requirements
reconciliation of disparate priorities
prioritization of requirements
preparation of requirements specification
validation of requirements specification

And for each of these tasks, we could, in turn further enumerate
and refine lists of sub-tasks. Most large software development
organizations have such process specifications. They may spell out

project phases
activities that are expected to be completed during each phase.
tasks and sub-tasks that are included in each activity
what outputs are expected from each activity
project approval decision points,
and how the decisions are to be made.

Many people consider these specifications to be very helpful, as
they guide the project team from one step to the next, and clearly
spell out what is expected of them at each point. Other people
consider such specifications to be meddlesome micro-management,
focusing on forms and process rather than working software.
There is validity to both views. More enlightened processes
ensure that the important things are covered while giving
projects wide lattitude in the interpretation of requirements
and means of demonstrating satisfaction.

4.1 Checklists

It may be tempting to thing of check-lists as mnemonic aids for
people who have not yet mastered the concepts. This would be a
mistake.
Jet Fighter pilots are some of the smartest, proudest, and highly
trained people in the world, and they depend heavily on check-lists.
They do not use check-lists because they haven’t been adequately
trained, or haven’t yet memorized the procedures. They use check-lists
because they perform complex tasks, where any mistake is
likely to result in disaster. In times of emergency, when response
time is most critical, they still adhere rigorously to their check-lists.
Check-lists are for people who can’t afford to make mistakes.

Most software process definitions, and most software engineering texts,
contain numerous check-lists: things that you should make sure have
been covered. There is considerably more variation among software
projects than there is in flight operations, which means that these
check-lists are not as comprehensive, and may contain steps that are
not applicable to all projects:

it is easy to say “not applicable” as you run your finger
down the check list.
if you find important considerations that are missing from
the standard check-lists, create your own.

Don’t gloss over those minute details, or resent the seemingly endless
lists of items to check. Every item on those lists is there because
smart people have forgotten about them, and come to bad ends as a result.
Check lists represent a great deal of collected wisdom. Using them will
save you a great many mistakes, and make you a much better engineer.

4.2 Process Taxonomy

phase
broad characterization of the evolutionary state of
a process. Typical phase names (e.g. definition,
planning, construction, deployment) have been discussed,
but different organizations can define different sets
of phases for a project. The real meaning of a phase
is best understood by the activities it encompasses.The transition from one phase to another is often
accompanied by a review and approval.
task
a well defined assignment, involving enumerated steps,
to be carried out by a specific person or group. A
task might be the development and execution of a
unit test suite for a particular module.
activity
general characterization of work, that may encompass
a great many tasks: unit testing might be an activity
within the construction phase.Some people use the term task to refer
to something that has a clear beginning and end (e.g.
reading this article), whereas an activity
(e.g. learning software engineering principles) may
continue indefinitely.
work-product
any process-defined output of a task. They might
be primary products (e.g. code), key inputs to other
tasks (designs), or evidence of compliance (review
or test reports).Any work-product definition must specify the
required content and qualities, and many organizations
specify standard forms for each standard work product.
Many people believe that all specified work products
should be inputs to subsequent process steps.
- don’t specify it unless you plan to measure it.
- don’t require it unless you plan to use it.

5. Conclusions

An understanding of the basic process models and taxonomy enables
us to more intelligently plan and discuss projects. An understanding
of the relationship between abstract software process models and
real software processes keeps those discussions real (useful).