The importance of software quality attributes

Software architecture is hard. It is a never-ending process of decision-making and tradeoffs.

Each time a change arrives it has an impact on the software. This impact might be huge or very small, but it is inevitable. For example, Netflix had decided to move from monolith to microservices which had an enormous shake on the whole infrastructure. On the other hand, each new line of code introduces a small effect. Yet when time shifts, the accumulated lines become chaos with unpredictable impact, without proper safeguards.

There are many good practices to mitigate this. The most common are tests, CI/CD, canary releases, and ADR, to name a few. In combination, they give some degree of confidence when introducing changes to a system.

Yet, when architecture is a key ingredient in an organization’s success, the system must be driven by a set of quality goals. For example, if a system provides behavior-based security for new customers, it gives access to its functions and supports through websites, APIs, etc. But to be successful, it also must work within restricted performance, availability, security, reliability, and supportability. If the system and architecture fail to do so, then the business probably goes down.

When it comes to reasoning about software quality attributes, things are more complicated. What is the software quality attribute anyway? Does each software is characterized by the same qualities? Does it even matter to have quality attributes defined? What happens when there are no quality attributes?

This article tries to answer mentioned questions.

Software quality attributes

A quality attribute is a property of a software system that determines its usefulness and success from the client and business standpoint. For example, if a ledger system is designed with technical excellence but is hard to use, doesn’t have support, and has a long time to market for new features, then it probably fails in the long run.

A quality attribute acts as the driver for a software system. It defines constraints and goals that the system architecture, development process, and the whole Software Development Life Cycle (SDLC) must fulfill; otherwise, the team responsible for an application might get lost.

But are there any software systems without QA defined? The short answer is no. The long answer is that there are two types of systems. Type one, has well-defined and known quality attributes which drive informed technical decisions, and the whole rest of the systems. This article gives examples of both in later sections.

Classification

Image source 

Software Quality Attributes (QA) classification is defined by the standard ISO/IEC 25010. There is also an extensive list of QA at Wikipedia. A detailed description of each quality attribute is outside the scope of this article. All interested readers may refer to the mentioned standard.

What is the value added from defining software quality attributes?

From the classification of QA, it may be concluded that quality attributes are the driving force for a system by giving a direction to which this system should evolve. They also provide constraints to the inevitable changes to the system. Well-defined characteristics allow developers to reason about each change. For example, if security is a highly important part of the system and it says that ‘all connections between services must use OAuth2 authentication’, then each service must comply with this quality attribute otherwise it won’t be deployed to production.

Typically quality attributes should be derived from business needs. Extracting and defining those attributes is a hard part, because business needs may change over time. But for this reason, it is worth putting effort into defining them. As business needs are changing we as developers may use QA to question or validate if a given change is reasonable. When a system has a set of quality attributes supported by metrics, then it is much easier to tell what is the potential cost of change, what is the impact, what is the risk, and how to handle it. Additionally, when money is involved in the discussion then it is also easier to reason about trade-offs.

Evaluate software quality attributes

A single quality attribute ideally should be represented by a discrete and measurable value. The measurement tests should be automated and effortless to run. It is rarely possible to have a single quality attribute represented by a single value. In most cases, it is composed of multiple values, and its complexity depends on the business needs. But how to distill quality attributes, quantify them and apply them to system architectures?

Imagine a hypothetical company WorkoutPlans that is about to start a business highly dependent on software. The idea is to create a service that supports athletes in their workout plans so they can effectively prepare for the competitions.

Service should be accessible as a gamified application with 3D visuals for bicycle riders, runners, and kayaking using training equipment connected to a PC, or laptop. Service must support smart equipment with controllable resistance depending on the virtual environment.

In the future, it should be possible to support any new equipment that will show up on the market.

The service must accurately calculate all the values necessary for athletes to reason about their fitness progress. The service must be available 24/7.
It must withstand at least 40000 active users. It must be available for different devices – PC, Apple MacBook, Apple Desktop, IPad, and Android tablets. Finally, it must be easy for users to set up a workout plan, create their workout, do non-structured training, navigate through settings, view their progress, connect with other athletes, and integrate with external fitness services.

Considering this short business description a few quality attributes may be extracted.

• Reliability – users must be able to access the system at any time
• Performance – the system must withstand at least 40000 active users
• Security – multiple user’s data must be secured by using authentication and authorization
• Portability – the system must be available on many platforms
• Maintainability – the system must be open for updates
• Usability – the system must be usable for casual and advanced users
• Functional Suitability – the system must accurately calculate fitness metrics based on the data collected from training equipment

From the business point of view, this set should give an insight into the system characteristics and what to expect from said system. It should be relatively easy to reason with stakeholders at this level. For example, the system architect can decide together with the person of interest, if Portability is important for success considering the effort and time required. It might be that Portability can be excluded or minimized depending on needs. On the contrary, Portability might be crucial when taking the target market into account.

“Observability is the ability to measure the internal states of a system by examining its outputs. A system is considered “observable” if the current state can be estimated by only using information from outputs, namely sensor data.”

Quote source 

System observability is a crucial part of determining quality attributes. Without observability, one can’t reason about the current state of the system, and quality attributes in particular. Without sensors exposed by the system, it is hard to tell if there are any problems with traffic, performance, communication, broken connections, database failures, etc.

Yet, a system architect needs to know what to observe before deciding on architecture. Otherwise, the shape of the solution would end up with a best guess and knowledge at that moment.

Introducing quality attributes gives the possibility to target observability from the starting point. It allows architects to focus on the goals that a given system should aim for. For example, in the domain of civil engineering, it would be tough to construct a bridge that withstands predicted loads without knowing what this load is. This knowledge has an enormous impact on the final project and construction. It is analogous to software engineering, but with much less dramatic consequences.

With quality attributes extracted from the example with WorkoutPlans company, it is possible to define measurable values that tell what to observe exactly. The table shows discovered quality attributes with descriptions and definitions of values in the ‘Invariants’ column. Those values should direct architecture design. They are chosen arbitrarily as an example.

This simplified process of distilling and defining quality attributes is an example of how to lead architects closer to informed decisions. The decisions the architect makes in the early stages are the most important because the architecture starting point has an enormous impact on the rest of the decisions that will be taken in the future. The closer the architecture to the business needs the less probability of issues on IT – business intercourse, and the fewer missed significant decisions on the architecture level.

An architect can plan disaster recovery strategies, scalability strategies, security policies, etc., based on the said characteristics. There is less room for guessing what needs to be done.

Architecture with unknown QA

A company making business in the loans domain is highly dependent on the underlying software. The system calculates loans, does money laundering risk assessment, tracks existing loans, manages customers’ accounts, and many more. In other words, without this system, the company would have to do all those things by hand or using excel. Thus, the system is the core part of the business.

However, a decision has been made to put the whole IT department, along with the system, into the cost budget instead of the investment budget. That had a huge impact on all the decisions and actions related to the system.

The only real driving force for the whole company is to make more money with the cost as small as possible. And that is good. Fundamentally this is the essence of the existence of all companies.

One might say that this system has one quality attribute defined, namely cost minimization. Due to that one single characteristic a bunch of architectural decisions are made:

• System observability is limited to logs only
• No security policies
• Single monolith deployment for the system that supports different countries
• Single shared database for supporting different countries
• A common library for calculating all the use cases for different countries
• Introduced release train for all teams working on the countries’ components
• The single code base for all teams
• The single testing department dedicated to all teams

Photo by Ricardo Viana on Unsplash

What a mess

Those decisions are made unconsciously as an effect of company management, one after another as time went by and the complexity grew.

In consequence, the system becomes unreliable, hard to maintain, hard to change, and hard to diagnose. Additionally, there is no place and time for making significant changes that would fix the state of the system, because it is hard to allocate those changes to the cost budget. With no surprise, the real cost of maintaining the system grows over time.
The following are the main factors that have an impact on the overall cost:

• Bug fixing takes time due to a lack of observability
• Bug fix delivery depends on release train frequency
• Feature delivery depends on release train frequency
• A single code base rises the probability of unintentional side effects
• Testing all aspects of a system by a single testing team takes enormous time and effort. It is also prone to errors.
• Database fixes take time due to shared database and lack of observability
• A common library for processing loans for all countries is hard to maintain due to many different scenarios. It is also prone to errors.

On top of that the teams working on the system are getting more frustrated and stressed, especially during the release time because, if a particular team won’t make it on time then they miss the release window. Because of said issues, the business loses money and loses customers, and that leads back to the starting point – the foundations of this system are based on cutting the costs of the IT department.

Summary

Software engineering is a very dynamic environment. Requirements change, problems pop up, and business needs evolve. Everything is fragile. Change is the only constant thing.

Uncertainty and complexity are integral parts of software systems. Software engineers must cope and navigate through this dynamic world. Usually, decisions arrive by best guess or gut feeling. In the beginning, this approach seems to be sufficient, because the complexity of the system is reasonably simple to grasp. Yet inevitable changes incoming from different angles stress the system to the point that it is unstable, and hard to reason about and maintain. Business losses and people are getting more frustrated over time.

On the other hand, making the decision process more analytical and predictable is challenging, and it seems to be too much of a cost of time and funds. Quality attributes of the software system are one way to help overcome this challenge and make it easier to digest. They help to define the system goals and as a starting point drive system architecture in an informed manner leading to smaller costs in the long run.