Filing Good Bugs

Writing good bugs is important when dealing with software development. I deal with several Bugzilla installations meaning I’ve got some experience with both good and bad bug reports. While the specifics may vary based on the product your submitting a bug report for these basics rules will make life easier for everyone. In many cases will even help things get resolved quicker.

Bugs that are easy to triage, sort and sift are easier to take care of than those that are cryptic and require more investigation. If you want your issue to be addressed promptly make sure you facilitate the process by writing a good bug report. It’s not difficult and doesn’t require much technical know how, it just requires being explicit yet concise. If more information is needed a developer can request it and explain how to capture it.

Summary

A good summary is explicit yet brief. Upon reading it the purpose is clear and little is left to question. Here are several poor summaries, and the ideal way to submit them:

Bad: “add me to planet”
Good: “add john doe to planet.mozilla.org”. There’s no ambiguity there. Just reading the subject I can distinguish it from others in the queue.

Bad: “site crashes browser”
Good: “visiting a video permalink page crashes IE6”. Less ambiguity just make sure to add the url to the bug.

Bad: “javascript error”
Good: “Contact form has a javascript error in Chrome”. Again less ambiguity is key.

Description

Descriptions should ideally be front loaded with the important information. If it’s a bug report and not a request it should be followed by steps to reproduce including what you see, and what you expect to see. Reference attachments as appropriate (screenshots can be helpful).

Bad: “Visiting a video permalink page crashes IE6.”
Good: “Upon visiting a video permalink page in IE6 the video automatically starts playing. About 10 seconds into the ad the browser crashes. I am able to reproduce this consistently at the following link: http://domain.tld/…”.

Bad: “Contact form shows a JS error, can’t submit.”
Good: “When attempting to submit the contact form [http:/doman.tld/…] in IE 7 I get a “object expected” error and it refuses to continue.”

Always include URL’s when applicable or examples on ways to reproduce the error when possible. It’s hard to act upon “several pages” when your site has over a half million pages.

Attachments

Screenshots, traces, debug logs, mockups are generally helpful. As a rule make try to use browser-readable formats when possible. JPEG, GIF, PNG for images, txt, or if you must PDF for documents. Having to open PowerPoint or Word to view your screenshot isn’t cool. It makes people avoid what would be helpful information. Explanations on what you’re seeing and expect to see are also helpful for when it’s not obvious to someone else.

Severity/Priority

Always try to use restraint when setting the severity or priority on a bug. Unless it’s truly a show stopper avoid using the most severe status. Generally speaking most “normal” or a P3 (on a P1 – P5 priority scale). Remember The Boy Who Cried Wolf.

Version

If you know the version, please specify it. It helps when several versions of a product are still floating around.

Product/Component

I put this toward the bottom for a reason. Making a best guess is the most you can generally do. Just be sure to read the entire list before deciding as the best option may be beyond the one you’ve already found.

Following these simple guidelines can go a long way to making sure bugs get resolved quicker.

Help Wanted: reporter cleanup and bug filing

I could use a few volunteers to do the following:

Help me clean up reporter
Everything needs a cleaning now and then. Check for reports in reporter that just don’t belong. “Microsoft Sucks” isn’t a valid report. Some are merely tests. Find them and send me the reportID’s (you can leave a comment if you wish or send an email).

Help file bugs
We have a fair number of reports already. Start sifting and file bugs in bugzilla (under Evangelism) with good valid bug reports. Be sure to mention in the bug what reports found the bug, so we can have detailed info on the setup’s that involve this bug. Also please make it block Bug 301962 so that we can keep track of bugs based on reporter data.

Safari’s WebKit is now open

Hyatt made the announcement on his blog. Using bugzilla, and cvs. It says it’s even possible to get checkin access if your a contributor with a proven track record. I didn’t see any lxr like tool for viewing source via the web (very handy), or a list of reviewers. Perhaps that’s still to come, the website looks pretty new.

Anyway, congrats to the Safari team in addressing all those concerns a few weeks ago.

I’m curious how perhaps this can improve browser relations across the board. Not that the Safari team and Mozilla are so distant to begin with, but this could lead to bigger better things. This is great news for everyone.

Mozilla Reporter

History

As I recall Asa and I started discussing this back in July, evaluating what the ideal system would be, how it would help, who would use it, etc. etc. It’s evolved a bit from the original concept. Much more mature. With the extension as the input device, it’s collecting very relevant, useful, and accurate data. No more bad/incomplete/typo ridden bug reports in bugzilla.

What is it?

Simply put, talkback for gecko. A simple automated way to collect feedback.

What are the goals

  • Provide Agregate data on incompatible sites with Gecko browsers
  • Provide an interface for end users to submit problems they encounter in under 30 seconds
  • Keep end users away from bugzilla, and force them to enter good Evangelism reports
  • Provide a tool for layout folks to use to track problems that effect Gecko users most

Why can’t I edit/fill out fields in the form

The goal is to provide relevant correct data. Allowing users to modify data makes it less accurate (typo’s, incomplete, incorrect data). The tool gathers data directly from the browser itself, so we know it’s correct, and sends it to the server.

Privacy!

The tool only invokes when you run it. That is the only time. Never will it send any data without your consent.

We will likely use a random hash as an ID per computer, so we know if 1 user submitted a site 100 times, or 100 users submitted a site 1X each. We won’t bind it to any other info. Nobody will be able to view the data and say “Hey checkout what Bob is viewing”. The final version will put an option to enter an email address. Only privilaged people (by a process yet to be defined, most likely similar to that of cvs commit access) will have access to such data. It will be optional data. We capture IP address’s, but again that’s behind a password. No regular user can visit and get the information.

It’s ugly

The version on my personal computer is much nicer :-D. I will be deploying an update to the server sometime soon (no timeline yet), containing many updates/changes/enhancements. It takes into account Ben’s recommendations, among many other things. When I do, current versions of the extension will no longer function. You’ll have to download the new build then. Sorry, but the auto-update wasn’t implemented in the current one. Simply because I didn’t really anticipate many people toying with it.

What about bugzilla?

Bugzilla isn’t going away. It’s just getting better. Consider this Bugzilla’s personal secretary. That’s really what this is. Just like with trackback, confirmed popular problems will be entered as bugs by Mozilla Volunteers on Bugzilla, and treated appropriately. This just prevents a bizillion dups, and lets users help without needing buzilla accounts, or query bugzilla, or figure out what we need to know. In a few clicks, the perfect report is submitted.

when will it be done

Were making quick progress. No distinct timeline, but Asa and I have discussed the idea of an end being in sight, and making preparations. So that’s a good sign.

Feedback

Contact Me with any feedback you may have, or leave a comment. I’ll try to address them all in coming days, though I’m pretty busy with school, and this project.

Trolls and Bugzilla

Times like these, I wish I had the appropriate permissions on my bugzilla account to terminate an account. No Asa, you are not a “Hitler” nor a “cancer”.

In all honesty, from Bug Days on IRC, to setting up the new r.m.o system, Asa’s been great to work with. He’s really the easiest person in the Mozilla Foundation to get in touch with. A regular blogger also, to keep us all informed about what’s going on.

/me thinks it’s time for Bugzilla to have an email address on the bottom of every page to make it easy to report abusive posts, like many forums do. I suspect this will only happen more frequently as more end users attempt to use Bugzilla for support, or to complain, rather than as a bug tracking/project management (since it does do that) product.

I didn’t go with Bugzilla

I was looking for a good bug tracking system, for my personal use. Of course the first thing I turned to was Bugzilla. But I decided instead to go with Mantis. Here’s what I was looking for:

  • Streamlined UI, fast quick bug posting
  • Simplicity
  • Good Sorting
  • Good Control over data

Mantis has by far the better UI at this point. It’s a bit more intuitive than Bugzilla. But then again, bugzilla was designed by/for the developer. The pretty good use of HTML/CSS allows for me to get people to report issues with Mantis without me walking them through it, or tons of mistakes.

Simplicity. Bugzilla is much more complex (both good and bad, as I’ll get to in a moment). Mantis was simple.

Good Sorting. Bugzilla had a bit of an edge there, but the UI in Mantis made it easier to access than Bugzilla. Bugzilla has the better sorting, but Mantis has enough for me… for now.

Good Control over data. I think Bugzilla gets this one as well. But both are good.

In the end I went with Mantis. Since I primary use it as a way to organize/index issues with MacVillage.net, and other online activities, I’m the main person using it. A few people will on occasion be using it to provide some feedback on something, but it’s mostly myself. It’s my online organizer in a sense. I find it much more effective than a text based todo list.

So what do I recommend as a bug tracking system?

For open source projects, or medium to large projects, hands down bugzilla. It’s more robust, powerful, and flexible. It’s a great product, despite looking a bit ugly and unimpressive at first. There’s so much there. I’m sure eventually someone will come around and rework that UI a bit, to make it intuitive, and bring all the features to the surface. And it has gotten better in that regard over the years.

But for a small bugtracking instance, I have to recommend Mantis. While it seems like it would be capable of larger things (custom fields, LDAP integration, multiple projects etc.), I just don’t see something like Mantis managing something as large as Mozilla’s Bug database… but then again, it’s sub 1.0.

Is Bugzilla bad? No, just not perfect for all jobs. I think it’s the perfect tool for Mozilla, SpamAssasin has a nice instance of it as well. So does Apache. Great for them. But for a shareware developer, or some other small time gig. I’d have to push Mantis at this time. It’s lighter, and better designed for such instances. No need for Bugzilla’s super powers, and Mantis’s UI allows for complete control over available facilities with ease.

This could be huge

Bugzilla is a ugly beast, but it’s a great, solid product. Never saw anything better.

Looks like a testcase manager may be on the way.

I think this could really help make things more efficient. Lets hope it all goes as well and this can become a new tool. Who knows, perhaps they can dump the code into Bugzilla and integrate it completely? Would be a great new feature. (pure speculation, haven’t seen it, or read much. No idea what the licensing is, and potential conflicts).

Regardless, sounds like a great method to enhance Bugzilla.

My own bug

I created a new about:buildconfig page. Something to mimic the about:plugins page. I then took my page and filed a bug. Only things I messed with were XHTML and CSS.

It’s mighty neat filing a bug and the code to resolve it. First time for me. Now I need someone to review, and put in CVS. Not to mention love my work and worship me for my silliness.

Why is this page better?

Consistent design – Matches the plugins page
XHTML Compliant – w3c validator certified. It’s good solid code.
Looks nicer – could even provide more info in the future in a much more organized, professional looking way.

So hopefully someone will review it soon. We shall see…