What developers need in a bug report: what the research says
A study of 466 developers and reporters at Apache, Eclipse and Mozilla found the most useful details are also the hardest for reporters to give. What that means for your bug reports.
Short answer: in the best-known study of bug report quality (Bettenburg, Zimmermann and colleagues, 2008), developers rated steps to reproduce, stack traces and test cases as the most useful information. Those are also the hardest things for the people reporting to provide. Good reporting tools close that gap by capturing context automatically and asking for what’s missing.
The study
- Developers and bug reporters from Apache, Eclipse and Mozilla were surveyed; 466 responded.
- Developers ranked which parts of a report help them fix bugs.
- Reporters said which parts they usually provide and which are hard to provide.
The mismatch
| What developers find most useful | Why reporters struggle to give it |
|---|---|
| Steps to reproduce | People remember the result, not the exact path that led there |
| Stack traces | Hidden from users; they don’t know where to find them |
| Test cases | Requires technical knowledge of the product |
What this means in practice
- 1
Capture what people can’t
Page or screen, browser or device, app version and a screenshot can be attached automatically. Don’t ask people for what software can collect.
- 2
Ask for what only they know
What they did just before, and what they expected to happen. Ask while the steps are still fresh, and keep an email follow-up available if the person has already left.
- 3
Keep the first step easy
A long form in front of the report means fewer reports. Let people write a line first.
This is how Bopmigo is designed: the context is captured for the person, and a short follow-up asks for the missing detail, usually what they did or expected, while they’re still there.
See also: how to write a good bug report (template) · vague bug reports from users