Top 5 Software Metrics to Manage Development Projects Effectively

Top 5 Software Metrics to Manage Development Projects Effectively

What are Software Metrics?

Software development metrics are quantitative measurements of a software product or project, which can help management understand software performance, quality, or the productivity and efficiency of software teams.

Types of Software Metrics

• Formal code metrics—Such as Lines of Code (LOC), code complexity, Instruction Path Length, etc. In modern development environments, these are considered less useful.
• Developer productivity metrics—Such as active days, assignment scope, efficiency and code churn. These metrics can help you understand how much time and work developers are investing in a software project.
• Agile process metrics—Such as lead time, cycle time and velocity. They measure the progress of a dev team in producing working, shipping-quality software features.
• Operational metrics—Such as Mean Time Between Failures (MTBF) and Mean Time to Recover (MTTR). This checks how software is running in production and how effective operations staff are at maintaining it.
• Test metrics—Such as code coverage, percent of automated tests, and defects in production. This measures how comprehensively a system is tested, which should be correlated with software quality.
• Customer satisfaction—Such as Net Promoter Score (NPS), Customer Effort Score (CES) and Customer Satisfaction Score (CSAT). The ultimate measurement of how customers experience the software and their interaction with the software vendor.

What is Software Quality and How Can Metrics Help?

Software quality is a measurement of something intangible, “how good” a software product really is. The table below shows four aspects of software quality taken from the CISQ software quality model, and which metrics can help quantify each one.

Quality Aspect
What it Measures
Relevant Software Metrics
Reliability
How stable is the software and the degree of risk of failure
Production incidents, average failure rate, load testing,  MTBF, MTTR

Performance
How efficient is the code, how optimal is the architecture, is the system able to scale, load times of pages or key functionality
Load testing, stress testing, soak testing, Application Performance Monitoring (APM)

Security
How likely it is that attackers might breach the system, interrupt it or gain access to sensitive information
Number of vulnerabilities, time to resolution, deployment of updates, number and severity of security incidents

Maintainability and code quality
How easy is the system to debug, troubleshoot, maintain, integrate and extend with new functionality
Static code analysis, code complexity, lines of code (LOC)

Of these four, performance is probably the one that is easiest to measure. The other three are tricky to quantify and while metrics exist, they do not show the complete picture.

Metrics in the Software Development Process

Metrics in Agile Software Development

The agile methodology has a unique approach to software development metrics and how to use them effectively. Leon Tranter presents three agile principles for software metrics:

1. Metrics should be used by the team—They should not be imposed by management, rather they should be useful for the team to assess and improve their own work.
2. Metrics should be part of a conversation—A metric is just a number, it should be combined into a deeper discussion about process and challenges faced by the team.
3. Metrics should be used as part of an investigation of the experiment—Agile teams should use metrics with a specific hypothesis in mind, not measure for the sake of measurement.

Agile Metrics vs. Traditional Software Metrics

In traditional waterfall software projects, it was assumed that software could be specified in advance and quantified by estimates. It was also assumed that the software specification would meet end-user requirements (which often was not the case).

This led to an emphasis on in-process measurements like man-months or active days, Lines of Code (LOC), and even number of pages of documentation. In agile development projects, the emphasis is on outcome metrics like story points completed, production defects or customer satisfaction.

Top 5 Software Development Metrics Examples

We chose the metrics below as our top 5 because they provide direct information about the experience of the end user: their satisfaction, the rate at which they receive new features and new versions, the number of bugs they experience, and the degree to which their important functionality is tested.

1. Customer Satisfaction

A widely used and respected metric for customer satisfaction is Net Promoter Score (NPS). It a number ranging from -100 (indicating no customers refer you to others) to +100 (all customers likely to refer you to others). SurveyMonkey provides the full calculation for NPS and lets you set up a free NPS survey.

Example: If you received 100 responses to your NPS survey with 15 responses in the 0-6 range (detractors), 25 responses in the 7-8 range (passives) and 60 responses in the 9-10 range (promoters), your NPS score is the percentage of promoters minus percentage of detractors, ignoring passives = 60—15 = 35.

2. Team Velocity

Team Velocity measures the number of story points, quantifying the number and size of product features, completed by the team in the previous sprints. It helps to understand how much value the team is providing to customers in a given time period.

Example: If a team completed 15, 20, and 13 story points in the last three sprints, the average velocity is 16. This is the expected output of the team in the next iterations. It’s important to only count story points towards velocity if they were “done-done” according to the team’s Definition of Done within the sprint.

3. Release Burndown

A Release Burndown chart can help you understand, across an entire software release, how development is progressing, how much of the planned software functionality remains to be done, and when you can expect the release to be completed.

Example: The image below, taken from Atlassian’s JIRA software, shows the power of the release burndown chart. It shows that over four sprints the team has reduced the number of stories to be done from 43 to 26, and predicts that the release will be completed in 7 more sprints.

4. Escaped Defects

This metric checks, per release or product component, how many bugs or issues were identified after the software was already in production. This is a good indication of quality as perceived by the end user. If it’s increasing, it is a sign of a faulty quality process.

5. True Test Coverage

We call this metric true test coverage as opposed to the regular test coverage metric, which only measures unit tests. This is a metric that tells you how much of your codebase or feature set is covered by all types of tests—unit, integration, UI automation, manual tests and end-to-end acceptance tests. It can reveal quality gaps—parts of the software that are new or actively used but do not have sufficient test coverage.

Using Quality Intelligence to Measure True Test Coverage: The Missing Software Metric

In our list of metrics above, you might have noticed that True Test Coverage is a new one which you might find difficult to compute on your own. In complex software projects with millions of lines of code and numerous test frameworks, collecting the data for a holistic test coverage metric is a major challenge.

This is where Quality Intelligence technology comes in—tools that provide visibility for development managers, by monitoring tests across all test frameworks, collecting test execution data, and correlating it with data about code changes and frequently used features. This can help compute a True Test Coverage metric which can expose quality gaps in a software product.

SeaLights is a quality intelligence platform that helps visualize test gaps and quality trends. For example, it can tell you if a software component suffers from quality risks, and where teams should focus to improve quality.

This is how SeaLights visualizes the number of methods in the code that were recently modified, or are used in production, and are not tested.

To learn more about the missing quality metric that can dramatically improve control and efficiency in developer projects, read our white paper.