The bug reporting mechanism

As all developers know, even the best of us leaves bugs here and there. I guess it comes with the package.
One of my teachers said that it was all a matter of probabilities : coding complex routines and structures gets exponentially difficult as the size grows, and therefore produces statistically more bugs. QED.

Most of us, when we start, think we produce bug-free software. The program is too small, the users are too few too even notice, and even if there are bugs, they are peripheral at best. So we just ship out the product and drift happily back to another project.

And the bug report hits us. It can be:

  • A bad review, online (cowards!)
  • A friend saying “you must have noticed this, that’s why I didn’t report it during the beta testing phase. Why did you even ship it?” (that’s what beta testing is about, dammit! I don’t see the same problems you guys see!)
  • An email stating the program crashes/loses data/does something really dreadful, without any detail (I know a bug sucks. I want to fix it too. Please help me!)
  • An email by a fellow developer, or a power-user, who actually details everything, including the crash log, along with some suggestions on the way to fix it (most of the times, this is very useful, but can be ill-perceived, depending on how condescending the tone feels)

Apart from the last one, the real problem lies in the “shyness” (let’s say) of the people reporting bugs : either they don’t want to be confrontational, or they think the bug is not a real issue, or they just assume someone else will report it. In my book, it’s a cultural thing. People would like to help, but they assume the worst.

We developers are not monsters, we don’t prey on poor users unable to defend themselves. Most of us aren’t even real geeks, incapable of interacting socially with our fellow human brothers. We just talk a different language. And we are used to interacting with a very very stupid entity : the computer.

As such, we need a way to allow non-technical people to grant us the benefit of their help that is as simple as possible for them, and as complete as possible for us. If there are technical guys reading, please bear in mind that explaining that is something that can also be interesting for you.

What do we need to make a good bug report?

  • A nice submission mechanism (email, web interface, crash reporter, …). The best ones are those that leave a trace
  • The setup of the machine. No need to go in deep details, just what’s relevant : OS version, computer model, and potential third-party upgrades
  • The actions that triggered the bug. If it’s repeatable (meaning “it doesn’t occur just once, I saw it several times in similar circumstances”), it feels really good to have a simple way to crash “on demand”
  • If possible, the crash log

When we have all these items, we are in debugging heaven. On small scale projects, emails are just fine. Even if you get 10 bug reports a week, you probably have a lot of duplicates. And here lies one of the major objections to bug reporting : “the developer must know already”. Well… probably not. And even if I do, it makes me feel this particular bug is more important than another. But it is a very good help to actually have a list of reported bugs somewhere accessible. That way people can look there to see if a report matters or not.

Personnally, I use RT ( from Best Practical ), for one reason : it is fully cross-aware between emails and web interface. When you modify something in the web view, it sends out emails, and when you send emails, it updates the web interface accordingly. It has all the gimmicks you want to use, such as severity and such, but it’s still pretty simple on-screen. It’s better not to frighten non-technical users with a flurry of options and popups.

Why did I ramble about bug reports exactly? Well, yesterday, I received a response to a bug I reported… in March 2005. Having a good bug reporting mechanism helps you keeping track of the time spent on bugs, and (if you are stubborn enough) it can nag you every so often to keep you from forgetting the “old ones”.

  

Leave a Reply