Quality and Testing

Bah, Scrum Bug! How to manage those bugs and issues

Posted by Ilan Goldstein on November 2, 2011

Although we no longer have to contend with actual moths infiltrating our vacuum tubes (yep, that’s where the term bug originates), their digital descendants are still regular visitors to every codebase on the planet. In the same way that bugs have changed over time, so has the way that they are dealt with, and this is particularly pertinent to our new agile way of thinking.

Previously, our waterfalling world viewed the handling of bugs very sequentially (see Figure 1).

Bugs in Waterfall

Figure 1 – A simplified flow of how bugs were sequentially handled in the waterfall world

With this simplified process as the benchmark, let’s look at what needs to change when implementing Scrum. In particular, it is important to appreciate that programming and testing need to be conducted in tandem rather than in sequential phases if teams hope to deliver working functionality early and often.

New Definitions

Before exploring some new Scrum-friendly processes for handling bugs, I’d like to set up some foundations with a few definitions and principles that I like to use.

Definition 1: Issues

  • An issue is a problem that occurs during the sprint and is tightly coupled to a user story that has not yet met its definition of done. Issues will therefore typically be picked up during the sprint (that the corresponding user story is being worked on) either by the programmer, the automated build, a tester conducting exploratory testing, or the product owner during a walkthrough
  • An issue is not a product backlog item (PBI). Instead, an issue should be seen as part of the evolving acceptance criteria for a user story. Essentially, what I am saying is that until the issue is resolved, the user story is not complete, and that being the case, an issue is a part of the actual user story rather than an independent, albeit associated, product backlog item.

Definition 2: Bugs

  • A bug is a bug only if it is identified after a user story has been completed and accepted by the product owner. Bugs will therefore typically be picked up by users (postrelease) or via an automated regression test (following the implementation of subsequent user stories).
  • A bug is a type of product backlog item. A user story is another type of product backlog item. Bugs and user stories should be prioritized together in the same product backlog and estimated using the same approach, such as relative estimation. A particular bug may relate to a specific user story, but it should be treated independently as far as any tracking and prioritizing is concerned. A bug can theoretically be represented utilizing the user story format, although I personally don’t find it to be a suitable format in most cases.

New Principles

Now let’s look at the three new principles.

Principle 1: Remove the Red Tape

Recall the second principle from the Agile Manifesto: “Working software over comprehensive documentation”. During my earlier years getting “soaked” in the waterfall world, I observed that a nontrivial amount of time was spent by both testers and programmers carefully documenting in painstaking detail the relevant bug minutiae. I remember regularly asking myself whether it was actually taking longer to document or to fix the damn bugs. Scrum relies on as much real-time communication as possible (rather than formalized, written bug reports), but if documentation is required, it should be fit for purpose and kept to the bare minimum.

Principle 2: Address Issues Immediately

There is nothing worse than your own stale code. Oh, wait a minute, yes there is — someone else’s stale code! Sadly, back in the day when we all used to follow the sequence detailed at the beginning of this article, it was very common to have to return to work on bugs in code that we had well and truly moved on from. The spin-up time to get back into the headspace to address old issues (be they yours or those of a colleague who’s off on vacation) is significant and, frankly, a big fat waste of time. The sooner an issue is found, the cheaper it is to fix, and that is why with Scrum, testing is completely entwined with the programming.

Principle 3: It Ain’t Over ’Til It’s Over

Bottom line is that unless a user story meets the definition of done, it might as well not exist as far as the customer is concerned. Customers are simply interested in final results and obtaining business value. If a user story is not yet done, it needs to be the top priority for the developer(s) working on it, and they should avoid moving on to any new work until it is completely done and dusted!

New Approaches

With our new bug-handling foundations now established thanks to the previous definitions and principles, let’s focus on some approaches that I recommend you follow within the sprint:

  • Scenario 1: A tester is conducting some final exploratory testing on a user story and discovers an issue.
    First, because the user story should be the top priority of the programmer working on it (see Principle 3), the tester should feel free to walk over to the programmer and explain and/or demonstrate the issue as soon as it is found. Again, because the user story is the top priority, the programmer should drop whatever he or she is doing and immediately jump on the issue. In this situation, there isn’t any requirement for written documentation, as the verbal discussion will suffice, assuming that the issue is immediately resolved and verified.
  • Scenario 2: Same as Scenario 1, but this time, the programmer in question is already in the middle of resolving a different issue (related to the same user story).
    In this case, the tester, after finding another issue, looks over and sees the programmer with headphones firmly in place and in the zone fixing a previous issue. The last thing the tester wants to do is to disturb the programmer mid-fix. As such, it is important to capture the details somewhere so that the tester can continue with the exploratory testing without forgetting the details of the new issue.As discussed in the definition of issue, an issue should be treated as part of the acceptance criteria of the user story, saving the tester the pain of creating a new bug, classifying it, assigning it, prioritizing it, and so on. Instead, my recommendation is that the tester simply add a line item to the acceptance criteria with a date/time reference, initial it, and add some bullet point details of the issue. When the programmer is free, a discussion can take place using the notes as a prompt. Also, the documentation ensures that the programmer can get on with the resolution even if the tester is not around for whatever reason.
  • Scenario 3: During the final user acceptance testing for a release, a range of trivial user interface bugs are found that were somehow missed during development.
    Again, let’s try to reduce the time spent on unnecessary administration; in this situation, I recommend that a single PBI be created as a container for the collection of minor bugs. Each specific fix may take only minutes, so creating individual PBIs for each issue could end up taking longer than the actual fix-ups!I recommend following this approach only if:

    • The trivial bugs are of a similar priority level.
    • They are somewhat related and it makes sense to tackle them at the same time.

    If these conditions aren’t met, then simply create separate PBIs for the items even if they are seemingly trivial in their own right.

    • Scenario 4: During a sprint, a critical bug is found in production requiring some of the Scrum team to resolve it.
      The first question to ask is, How critical is critical, or more specifically, can it wait until the next sprint? The last thing you want to do is to change the goal of the sprint. Assuming that the production bug can wait, it should be captured as a PBI, entered into the product backlog, prioritized by the product owner, and tackled potentially in the next sprint planning session.However, what should happen if the discovered product bug is one of those dreaded villains that simply can’t wait? Well, we then need to ask another question: How long will it take to fix the bug? If you recall from the article about sprint planning, it isn’t wise to max out team capacity for working exclusively on new sprint tasks to provide some room for handling non-project-related disruptions. As such, this buffer time can also be allocated to resolve the occasional emergency bug without disrupting the sprint.If, however, the resolution will take longer than the buffer time, you have two choices. First, you can treat these issues as impediments and track them accordingly (check out this article about Incriminating Impediments), or, if the issue is such a major drama (to the extent that it completely destroys the sprint goal), there is always the undesirable fallback position: a sprint cancellation that can be called by the product owner. A cancellation will end the current sprint and send the team back to sprint planning.

Turning Moths into Butterflies

Bugs can certainly cause pain, and like it or not, they’re never going to become an extinct species. However, what we have learned are better ways to deal with them. We now know that disposing of fresh bugs is easier than having to deal with old, festering ones and that spending unnecessary time documenting every issue is a waste of time.

Scrum handles testing and bugs very differently from traditional approaches. By adopting these new definitions and principles, you will start to avoid the unnecessary overhead and communication breakdowns that have previously stopped teams from turning those moths into less ugly butterflies.


If you liked this article, you can:
Subscribe to this RSS feed!

Find out more about managing bugs and issues in Scrum by taking one of our CSM training courses.



Sign up to receive our eNewsletter

Google Rating
Based on 325 reviews