Traceability Metrics

What is a Traceability Matrix?

The focus of any testing engagement is and should be maximum test coverage. By coverage, it simply means that we need to test everything there is to be tested. The aim of any testing project should be 100% test coverage.
Requirements Traceability Matrix to begin with, establishes a way to make sure we place checks on the coverage aspect.  It helps in creating a snap shot to identify coverage gaps.

How to Create a Traceability Matrix?

To being with we need to know exactly what is it that needs to be tracked or traced.
Testers start writing their test scenarios/objectives and eventually the test cases based on some input documents – Business requirements document, Functional Specifications document and Technical design document (optional).
Let’s suppose, the following is our Business requirements document (BRD): (Download this sample BRD in excel format)

The below is our Functional Specifications document (FSD) based on the interpretation of the Business requirements document (BRD) and the adaptation of it to computer applications. Ideally all the aspects of FSD need to be addressed in the BRD. But for simplicity’s sake I have only used the points 1 and 2.
Sample FSD from Above BRD: (Download this sample FSD in excel format)

Note: the BRD and FSD are not documented by QA teams. We are merely, the consumers of the documents along with the other projects teams.
Based on the above two input documents, as the QA team we came up with the below list high-level scenarios for us to test.
Sample Test Scenarios from the Above BRD and FSD: (Download this sample test Scenarios file)

Once we arrive here, now would be a good time to start creating the requirements traceability matrix.
I personally prefer a very simple excel sheet with columns for each document that we wish to track. Since the business requirements and functional requirements are not numbered uniquely we are going to use the section numbers in the document to track. (You can choose to track based on line numbers or bulleted-point numbers etc. depending on what makes most sense for your case in particular.)
Here is how a simple Traceability Matrix would look for our example:

 
The above document establishes a trace between, the BRD to FSD and eventually to the test scenarios. By creating a document like this, we can make sure every aspect of the initial requirements have been taken into consideration by the testing team for creating their test suites.
You can leave it this way. However, in order to make it more readable, I prefer including the section names. This will enhance understanding when this document is shared with the client or any other teams. The outcome is as below:

Again, the choice to use the former format or the later is yours.
This is the preliminary version of your TM but generally does not serve its purpose when you stop here. Maximum benefits can be reaped from it when you extrapolate it all the way to defects.
Let’s see how.
For each test scenario that you came up with, you are going to have at least 1 or more test cases. So, include another column when you get there and write the test case IDs as shows below:

 

At this stage, the Traceability Matrix can be used to find gaps. For example, in the above Traceability Matrix you see that there are no test cases written for FSD section 1.2.
As a general rule, any empty spaces in the Traceability Matrix are potential areas for investigation. So a gap like this can mean one of the two things:

1. The test team has somehow missed considering the “Existing user” functionality.
2. The “Existing user” functionality has been deferred to later or removed from the application’s functionality requirements. In this case, the TM shows an inconsistency in the FSD or BRD – which means that an update on FSD and/or BRD documents should be done.

If it is scenario 1, it will indicate the places where test team needs to work some more to ensure 100% coverage.
In scenarios 2, TM not just shows gaps it points to incorrect documentation that needs immediate correction.
Let us now expand the TM to include test case execution status and defects.
The below version of the Traceability Matrix is generally prepared during or after test execution:

Download requirements traceability matrix template here: Traceability Matrix in excel format

Important Points to Note About Traceability Matrix

The following are the important points to note about this version of the Traceability Matrix:
1) The execution status is also displayed.  During execution, it gives a consolidated snapshot of how work is progressing.
2) Defects: When this column is used to establish the backward traceability we can tell that the “New user” functionality is the most flawed. Instead of reporting that so and so test cases failed, TM provides a transparency back to the business requirement that has most defects thus show casing the Quality in terms of what the client desires.
3) As a further step, you can color code the defect ID to represent their states. For example, defect ID in red can mean it is still Open, in green can mean it is closed. When this is done, the TM works as a health check report displaying the status of the defects corresponding to a certain BRD or FSD functionality is being open or closed.
4) If there is a technical design document or use cases or any other artifacts that you would like to track you can always expand the above created document to suit your needs by adding additional columns.
To sum up, a requirements traceability Matrix helps in:

1. Ensuring 100% test coverage
2. Showing requirement/document inconsistencies
3. Displaying the overall defect/execution status with focus on business requirements.
4. If a certain business and/or functional requirement were to change, a TM helps estimate or analyze the impact on the QA team’s work in terms of revisiting/reworking on the test cases.

Additionally,

1. A TM is not a manual testing specific tool, it can be used for automation projects as well. For an automation project, the test case ID can indicate the automation test script name.
2. It is also not a tool that can be used just by the QAs. The development team can use the same to map BRD/FSD requirements to blocks/units/conditions of code created to make sure all the requirements are developed.
3. Test management tools like HP ALM come with the inbuilt traceability feature.

An important point to note is that, the way you maintain and update your Traceability Matrix determines the effectiveness of its use. If not updated often or updated incorrectly the tool is a burden instead of being a help and creates the impression that the tool by itself it not worthy of using.

Usage:

 1. Consider a scenario where, the client changes the requirement , something so usual in the practical world and adds a Field Recipient name to the functionality. So now you need to enter email id and name both to send a mail
2. Obviously you will need to change your test cases to meet this new requirement
3. But , by now your test case suite is very large and it is very difficult to  trace the test cases affected by the test cases
4. Instead , if the requirements were numbered  and were referenced in the test case suite it would have been very easy to track the test cases that are affected. This is nothing but Traceability
5. The traceability matrix links a business requirement to its corresponding functional requirement right up to the  corresponding test cases.
6. If a Test Case fails, traceability helps determine the corresponding functionality easily .
7. It also helps ensure that all requirements are tested.