8 Important Software Testing Techniques

In this article, I will explain important software testing techniques which help you during test and User Acceptance Testing (UAT) processes.

Disclaimer: The knowledge in this article is based on my experiences, understanding, and observation. If you don’t agree with anything in the article, please write a comment, and we will discuss the thing you disagree with.

What is User Acceptance Test (UAT)?

Product Owners (PO) and business units generally work on requirements and focus on them. Thus, they can use the requirement-based and activity-based test techniques during the UAT process. As testers, we can also use these techniques in the test process. I am going to explain some of these techniques in this article. But first, let’s start with what UAT testing is.

In agile methodologies, it is the test activity performed by product owners generally after the completion of the stories’ development and software testing process. In waterfall and V-model processes, these tests are generally performed by analysts or business units. The purpose of the UAT is to determine whether the user’s requirements for the requested job have been met and to get approval for the live deployment.

A sample entry and exit criteria for the user acceptance test are summarized in the following table.

Entry Criteria

● Code Written
● Code Reviewed
● Unit Tests Executed and Defects Fixed
● Tests Executed
● Resolved all High and Medium Priority Levels of Defects
● Prepared Test Data to be used in UAT

Exit Criteria

● All UAT Scenarios Executed
● All Acceptance Criteria Tested
● Reached Target Quality Level (> = 90%), (Passed Test Cases / Passed + Failed Test Cases)
● Defects Resolved (Priority> = Major)

1) User Story Testing (AGILE)

A user story can be described as a requested feature in the software from the end user’s perspective in the agile software development life cycle. In the user story, we have to specify the demand, the reason, and the user requesting it.

Definition of Done (DOD) defines the completion criteria such as code is done, a unit test is done, testing is done, UAT is done, etc… and Scrum guide states that the Scrum team (developers, testers, PO, etc.) owns and responsible for DOD.

Also, acceptance criteria should be expressed clearly by POs (the development team may help PO), and at least one test scenario for each acceptance criteria should be prepared in the test process of a user story. These acceptance criteria must be tested carefully.

Before starting the test run, the test entry and exit criteria must also be defined. Here is an example.

Test Entry Criteria

● Does the requested user, the need, and the reason of necessity are clearly understood?
● The risk associated with the demand specified in the User Story?
● The impact analysis specified in User Stoy?
● DOD’s acceptance criteria stated clearly?
● Non-functional requirements identified with expected metrics? (Performance, security, etc.)
● If there are integrations of the development, are they specified?
● Development Done?
● Static Code Analysis Done?
● Unit Tests were written and defects fixed?
● Code Review?
● Test Scenarios Written?
● Test Cases reviewed by the PO / Analyst?
● Critical tests reviewed and executed with Developers in Dev Environment? (Desk Check)
● Test environment prepared for testing?
● Data required for the test preparation in the test environment?
● Database changes transferred to the test environment?
● Configurational settings applied to the test environment?
● Prerequisites identified before starting the test?

Test Exit Criteria

● All Test Scenarios Executed and All Acceptance Criteria Tested
● Functional and Non-Functional needs were tested.
● Target Quality Level Meet (> = 85%), (Passed Test Cases / Passed + Failed Test Cases)
● Defects Fixed (Priority> = Medium)

Sample User Story

User Story 1:

As a Product Owner [User], to advertise Kariyer.net welcome campaign [reason of request], I would like the advertisement banner to be added to the top banner section of the Kariyer.net homepage [request].

Risks:

Homepage speed may be reduced
A bug in the animation of the banner will affect the homepage appearance
Continuous deletion of cookies can cause the banner to be continuously visible on the user side.
The function of the Close Banner icon is critical. It should work continuously and successfully.

Impact Analysis:

The banner loading function may be affected in the Admin panel.

Definition of Done:

Code Writing Done
Code Review Done
Unit Testing Done
UAT Done

Acceptance Criteria:

When Kariyer.net homepage is opened, the top banner is displayed as 200×200 for 8 seconds, and then it should be seen as 60×60.
When the user clicks on the banner, it should be directed to the Welcome page.
If a user visited Kariyer.net more than four times from the same computer, the AA-kobiBannerClosed cookie value should be four and more, and the banner should not be displayed.
The top right corner of the banner must have a cross-shaped closing icon, and the banner must be closed when clicked.
If the user previously turned off the banner, it should not be displayed again.

test techniques

2) Use Case Testing

A use case defines a user/actor’s operations in the system to achieve a specific purpose. Functional requirements of a system can be defined and managed using use cases. This way, the scope of the desired or requested job is determined. Test scenarios are prepared by considering the inputs and outputs of the steps determined by the user to reach a specific purpose. During the tests, the results of the tests are determined by comparing the expected outputs with the actual outputs.

When writing Use-cases, generally, business language is preferred instead of technical language. Therefore they are often used in writing acceptance tests. At least one test scenario is prepared for each requirement to cover all requirements. In this way, test coverage can be increased, and we can also measure this coverage using a traceability matrix. In the traceability matrix, we create a matrix table with test scenarios and requirements and put a cross sign in the relevant field if it meets the requirements for each test case. The goal is to cover all the requirements.

Scenario No

S.1

Scenario Name:

Changing Login Password

Connected Requirements

R.4

Affected Screens

Home Page, Login Page, Account Page

Prerequisites

An existing user on the system

Normal Flow:

1. The user opens the Kariyer.net homepage and clicks on the Member Login button.

2. log in to the site with the existing username and password on the login screen.

3. Click the “My Account Settings” in the pop-up menu.

4. Click the “Change Password” link under the “Personal Information Settings:” section on the Account Settings page.

5. The old password is entered in the password change section.

6. Then, enter the new password.

7. When entering the new password, the security level on the left side is displayed as three levels (weak-medium-strong).

8. Enter the new password twice as medium-strong and click “Save.”

9. At the end of this process, the password change operation is expected to be performed successfully.

Alternative Flows:

1. If the current password is entered incorrectly, you will be warned: “You entered your password incorrectly!”.

2. If a weak password is entered, you will be warned: “You can not change your current password with a weak password.”

3. You can not change the password by blanking any or all password fields.

Usage Predictions:

It is a scenario that can be used intensively by users. The functionally correct operation is critical.

Hypothesis:

Very complex password requirements can prevent us from being user-friendly.

Notes:

Sample Test Case

For test cases, you can use excel. For the sake of an example, all areas of the first successful test case are shown below.

Test Name

Changing Password Successfully with (Moderate Complexity)

Test Steps

Open Home Page.
Click Login Button
Go to Profile and click on “Account Settings.”
Click “Change Password.”
Enter the current password and new passwords (medium).
Click the Save button.

Prerequisites

This can be done with an existing user in the system.

Test Data

User Name: [email protected]
Current Password: kariyer1234+ New Password: asdf1234

Test Priority

High

Expected Results

It is expected that the password will be changed successfully, and the information message “Changed successfully” will be informed you that the password has been changed.

3) Checklist-Based Testing

In agile processes, we create a general checklist independently from user stories. If there is no risk specified in the User Story, all or some of these checklist items may be used according to the scope of the user story. During the execution of these tests, if you find a defect, you should extend the scope of the checklist by adding the failed scenario. Thus, we can increase risk items in the checklist for subsequent sprints.

Example

All links in the system (Web / Mobile) should work correctly.
There should not be a grammar error in the system writings.
Font sizes, and fonts, should be as expected.
There should not be any pictures that can not be loaded/broken in the system.
Pictures, text, etc. The alignment between the other components must be as expected.
All buttons must work properly, and each directs the user to the corresponding operation.
Each page should have the main page logo and be redirected to the main page when clicked.
Warning, information messages should be displayed in the correct format.
If the page is responsive, it should be checked at all resolutions.
All components on the site (dropdown, checkbox, radio button, etc.) should work correctly.
The input fields’ special conditions (numeric, alphanumeric, etc.) must be checked.
Operations can not be performed by leaving the required fields blank.
Any site operation must not last more than 3 to 15 seconds.
… etc.

4) Exploratory Testing

First of all, exploratory testing is not a random or ad-hoc test. One of the biggest misconceptions about this test technique is that exploratory testing is perceived as a random, non-testable, non-observable test technique. Exploratory testing is a test approach based on learning and exploring the product simultaneously using a test engineer’s experience, domain knowledge, and analytical and intellectual knowledge in agile processes.

Before starting exploratory testing, preparation should be done. Regardless of exploratory testing method selection, we should prepare a plan for the scope of functionality, tools to be used, test data, environment, etc. This plan will guide the tester during the test execution process. Another important point of exploratory testing is documentation is fully completed after the tests are finished.

Although it is not mandatory, the “session-based testing” technique is generally preferred as an exploratory testing technique. This technique involves the following steps:

Main Activities:

– Test Session Time (It should be a few hours)

– Session Activities

Session setup
Test Design and Test Execution
Defect Searching
Reporting

– The purposes of the test should be specified

– The goal of the test should be specified.

– Functionality included in the test (test report – charter) should be written.

Test Report During & After the Test Process:

– Test Report (Charter) [Specifies the test function.]

– The Person who Performed the Test

– Start Date and Time

– Session Metrics (Metrics Collected During Test Implementation and Defect Searching)

– Test Data

– Test Notes

– Results

– Errors

Recommended resources

5) Experienced-Based Testing

This testing technique is based on the knowledge, skills, and experience of the person who will do the test. In this testing technique, test planning, strategy, test inputs, and test scenarios are determined by the experience of the person performing the test. To prefer this technique, an experienced candidate with sufficient technical and business knowledge must perform this test.

It is easier to understand what is going right or wrong during the test because the experiences gained in past projects are considered. This person can perform tests using techniques like exploratory testing, making it easier to use past experience and intellectual/analytical knowledge. When we have a very short test execution time or lack sufficient documentation on the project, etc. makes sense to use this test technique. If the system being tested comprises high risks, it is not preferable to use the Experienced Based Testing technique alone because the context of the requirement should be covered.

6) User Journey Test

The User Journey test considers the various road maps and journeys of a typical user on the system. In these tests, the most critical journeys a user would make within the site are determined, and then the scenarios of these journeys are written. Thus, the user’s interactions with the system are covered as much as possible. These tests are usually “end-to-end” tests, so they may take more time to run than other tests, but the coverage percentage of these tests is higher than the others.

Since the “User Journey” tests are comprehensive and extensive, the numbers are relatively low compared to other tests. Especially, “user journey” tests can be created considering the most critical “use-case” scenarios. The most plausible behavior here is that positive basic scenarios, called “happy path”, should be run first.

For example, on kariyer.net, it is an important “user-journey” test for a user to enter the site, log in, search for the “sales representative” keyword and apply to the first job ad successfully. In this “user journey,” opening the site, logging in, making a successful call in the search bar, opening a related announcement page, and then submitting the application from this announcement page is covered. As you can see here, the broad coverage of the “user journey” tests is beneficial in the early detection of critical faults in the software development process, especially before starting extensive testing. Unlike User Story tests, user journey tests are not tied to user stories. When a new user story introduces changes, existing user journey tests are updated, taking these new changes into account. New user stories rarely cause new user journey tests. For this to happen, new features must be added to the system.

7) Risk-Based Testing

One of the most fundamental objectives of risk-based test techniques is to find the most critical and most important errors as early as possible with the lowest cost. Risks are things that we do not know exactly what will happen, but we know the probabilities of what might happen. Briefly, they are possible problems. When these possibilities are unknown, they are called uncertainties. Therefore, we can think of the general definition of the magnitude of the risk as the multiplication of the likelihood of problems and their impact. Thus, we prioritize and test the most error-prone functionalities in risk-based tests.

The magnitude of Risk = Likelihood * Impact

The early adoption of the risk-based test approach in projects is important for the early detection of critical problems.

The most basic steps of the risk-based test are summarized below:

1- First, risks are identified, and a prioritized risk list is prepared.

2- Make a test plan according to the prioritized risk list, and tests are executed for each risk.

3- As a result of the tests, some risks are eliminated, and some arise. New risks are tested by considering their test effort.

At this point, our most fundamental goal will be to find the most important defects.

You will need to do a detailed risk analysis if you are responsible for testing a product with a very high failure cost. Statistical models can be used here. One of the most known models is the Failure Mode Effect Analysis (FMEA).

In short, if we refer to the FMEA model, we calculate the risk score by taking three metrics with five scales. These:

Severity

Description
ImportanceLevel
Point

Data loss, hardware, or security issues
Urgent
1

Loss of functionality, not even have a workaround
High
2

Loss of functionality, which has a workaround
Medium
3

Partial loss of functionality
Low
4

Cosmetic or trivial
N/A
5

Priority

Description
Importance
Level
Point

Total loss of business value
Urgent
1

Unacceptable system business loss
High
2

Possible reduction in the business value
Medium
3

Acceptable reduction in the business value
Low
4

A negligible reduction in the business value
N/A
5

Likelihood

Description
Importance
Level
Point

A negligible reduction in the system value
Urgent
1

Some users are affected
High
2

Possible influences for some users
Medium
3

Limited impact of few users
Low
4

Impact not to be deducted from actual use
N/A
5

All these three qualities (Severity, Priority, and Probability) are calculated separately within themselves, then these values are multiplied together to obtain a Risk Priority Number (RPN).

Risk Priority Number (RPN) = S * P * L

Based on this RPN value, we can determine the scope of the test. Lower RPN indicates higher risk.

8) Heuristic Risk-Based Testing by James Bach

You can find the details at this link: http://www.satisfice.com/articles/hrbt.pdf

I just want to give a summary of Heuristic Risk-Based Testing.

At the beginning of your projects, your risk analysis may be incomplete or incorrect because it is impossible to estimate everything 100% at first. However, as your project progresses and your product improves, your estimates and risk analysis will strengthen. According to James Bach, the two most critical factors for risk are experiences and teamwork. Over a period of time, products or technologies begin to reveal characteristic problems. It is important to observe and learn about them. Besides, it is very critical to do risk analysis with people with different perspectives. Also, using risk-based lists helps us to negotiate with the management about test workforce effectiveness. We can show them that we use the existing test workforce for the most critical points of the product, or we may need to explain to them we need the extra workforce to cover all the risk areas. These risk-based lists give you extra power to negotiate with the management.

In this article, we covered very important test techniques. I hope you enjoyed reading it. All the things in this article are based on my understanding and experiences. If you disagree with anything, please do not hesitate to write a comment, and If you convince me, I will fix the thing that you are right about. 😉

You can also read my Q&A on Agile Testing Mindset article here:

http://www.swtestacademy.com/agile-testing/

Thanks.
-Onur

onur baskirt

Onur Baskirt is a Software Quality Leader with international experience in world-class companies. Now, he is a Lead Technical Consultant at Emirates Airlines in Dubai.