Wednesday, May 18, 2011

Writing Good Defect Reports

These are my ideas based on experience. I am writing this down to help you form your own ideas.
  1. Title includes the feature or area where the problem was found - The developer needs to know where you saw the problem. The person who retests the fix needs to know. Other testers need to know whether or not a bug has already been reported before entering a new bug. It would be nice to sort the bug list and see which areas have the most problems. The bug title BEGINS with location.
  2. Title describes the problem - Some people write defect titles with no problem description, similar to this: "Version 1.0.0> Home Page". It's even worse when they enter 5 bugs reports with the exact same title. Other testers have no way of quickly searching to see if the the problem has already been reported. The title should describe the problem so others can get a quick view of the types of problems that are being reported. The title should make the developer want to read it.
  3. The version number should not be included in the title - Bug tracking systems usually have another field where you can enter the version number where the defect was found. Besides, this issue may exist in multiple versions of the software. The version number doesn't make sense as part of the bug title.
  4. Steps to reproduce -  The steps to reproduce need to be repeatable. Get them narrowed down to the simplest form and write it down. Try following them yourself. You might find you are missing a key detail when you follow steps you have already written.
  5. Report relevant details - Don't include details unless they are relevant to the specific problem you are reporting. Include any preconditions that are required.
  6. Don't combine problems into one bug report - Each problem needs a separate bug report. You should create a new bug report if you are including more than one result. Each unexpected result equates to a new bug report.
  7. Ideas are not bugs - We are reporting results of an experiment, not writing opinions of how things should work. If expected behavior is documented in another document, reference the document and page number or reference a similar website, but we shouldn't report our opinion or ideas for improvements as a bug. Submit a feature request instead of a bug when you have ideas for improvement.
  8. Attach screenshots -  Highlight the section of the screen capture where you see the problem. The problem might be obvious to us, but other people have to decipher what we are talking about. Highlight the screen capture to save time. Attach screen captures as graphics files and not embedded in Word documents. It speeds up the process of reviewing the images and saves disk space.
  9. Include Expected Results - Describe what you expect to happen
  10. Include Actual Results -  Describe what actually happened
  11. Include supporting log files or databases - Attach evidence of the issue. Many errors depend on the data that was input to the application. Including a copy of the data used to generate the error sometimes helps narrow down the cause of the problem.
Which one do you think would get read and fixed first?

Version 1.0.0> Home Page
Home Page> company logo incorrect
Live Home Page> the company logo is currently being replaced with an offensive image

No comments: