At the very core of software testing is an analyst’s ability to identify and document defects. Most people can stumble upon defects in software given enough time and a desire to do so. However, it is more difficult to then take that issue and communicate it effectively to the individuals responsible for correcting it in a manner that will help facilitate the issue getting resolved.
Many times I have seen a critical defect go unfixed because it lacked vital information, misled the programmer on the importance of an issue or sent a programmer down the wrong path wasting time and energy and ultimately never getting fixed. All of which cost the development team time and money and put the product at risk of missing its ship date or releasing with issues that it didn’t need to be released with.
At Digital Dream Forge we have a standardized format for reporting defects that we have found has tremendously helped our clients to quickly track down and fix defects. This standardized method of reporting has been developed over the years and condensed down into just the necessary information to resolve an issue without being overly verbose and complicated.
STANDARD DEFECT FORMAT
When I first started in software testing I noticed that there was no standard being used when documenting defects. This of course was leading to a variety of problems for the development team so we standardized all of our defect reporting and immediately noticed an improvement in defect resolution. The format we use has six simple sections. All of which can be filled out quickly and help to resolve a lot of common questions a developer may have about a defect without them needing to ask.
- Expected Results
- Additional Information
This is a brief one sentence summary of the entire issue including what the defect is, what triggers the defect and where in the app the defect occurs. The summary should be able to stand on it’s own and provide the reader with almost all of the information necessary to understand and act on a defect. For a more detailed breakdown of how we write our summaries please check out our blog on the subject:
This is a list of steps that help to guide the reader through the process of recreating the issue. Starting with launch options all the way up to the observe step where we direct the reader to the screen location the defect occurs at.
This is an explanation of the defect and what occurs with the app when the defect takes place.
This explains what our software analyst was expecting to happen when taking the specific action that led to the defect. This is a critical part of the defect as it explains what should have happened instead of what happened. It also clearly allows the development team to understand why we thought an issue was an issue and not just an opinion.
Many clients ask why this field is so important. It is a valid question and one that can be easily summed up by the all pervasive defect classification “By Design” that you will see defect tickets being flagged with frequently. If we don’t list out what we expected to happen then the defect may simply be describing how the app was designed to work without explaining that the end user (The software analyst) didn’t know that and thinks there is an issue.
This part contains further information useful in tracking the defect down or explaining what is happening. It will frequently contain references to other defects or further details that help with explaining what we are seeing and what might be causing the issue.
This section will contain screen shots, crash logs, or other debug files necessary to help the development team to track down the defect and resolve it.
Attentive software testing professionals will note that I didn’t list things they are used to seeing in every defect like priority and severity. I didn’t list them because ultimately they are subjective and vary dramatically by development team. And frequently they are assigned by the development team as they are the ones who have to fix the issues. So from our perspective, while those things are important, they are typically handled by the development team or by a defined set of guidelines that the development team gives the test team when the project starts.
Digital Dream Forge’s standard defect format is very simple but very effective. It has been built upon decades of testing experience after working with hundreds of products and a large pool of development teams. It has proven to be very effective and helpful in quickly explaining an issue and providing all of the details necessary to get a defect fixed swiftly. And a quick resolution to a defect is, after all, the most important goal of all software testing.
By Jeremy S. Barnes