Recently on the Scrum list, I characterized the practice of having designated "QA" people on the team as a "second- or third-class practice." Here's some expansion on that ...

A Common Problem: Not Quite Done

One of the most common problems I see Scrum teams have is the inability to be “done” at the end of the Sprint. We often hear on the Scrum list about teams having a whole ‘nother Sprint to get things done. More commonly I just see teams who get most of their items “90 percent” done, or who have a high defect rate.

The Fix: Test the Software

The “fix” for this problem starts with testing. All the backlog items need to be tested within the Sprint, if they are to be done within the Sprint. (As I write this, I realize that one would think that this would be obvious. Apparently it is not.)

A Common Solution: Add Testers

One common solution is to add “testers” or “QA” to a team, and it can improve things. However, it results in an inevitable testing crunch, usually at the end of the Sprint. Since testing is hard to estimate, the results are not ideal. Things improve, but it becomes common for the testers not to be done, and there is even the infamous “QA Sprint” that some teams use to “clean up” the work of the preceding Sprint.

What's Going On

It turns out that what is going on is that testing is getting done mostly at the end of the development cycle of each backlog item, and that it is being done primarily by the “testing people”. This results in a number of ill effects:

  • Developers don't test very effectively.1
  • A testing queue builds up, slowing things down.
  • Defect fixing becomes a large and unpredictable part of the team's work.
  • The burn chart is incorrect to some unknown degree, resulting in uncertainty about progress.

A Better Approach

What seems to work better is to have the Whole Team focused on on each story being fully tested by the end of the Sprint, rather than letting the developers feel that they’re done and “waiting for QA”. Teams with this focus wind up devising some very productive approaches. For example, they’ll have their testing experts work early with the customers or product owners, to clarify up front how a backlog item will be tested. This builds greater understanding of what has to be done, which itself reduces defects.

Then, teams with a team focus on stories being tested will tend to focus on each story until it is done, rather than the developers screaming through as much code as they can, while testers struggle to keep up, followed by developers fixing bugs late in the Sprint or, worse yet, in the next one.

Finally, teams with this focus tend to develop much better automation techniques, as everyone is faced with the drudgery of manual testing, and everyone sees the value of repeating as many tests as possible, as often as possible.

Therefore, while adding testing skills to a team is a very valuable thing, I find that adding those skills by providing individuals whose job is thought to be “testing” is not the strongest move. A stronger move is to bring the whole team to the realization that no feature, and no individual, is done until the testing is done.

1 Poor testing by developers is commonly thought to be an inherent trait of developers, or to be a psychological effect due to the difficulty of effectively testing something one has created. The primary cause, I think, is simpler. If there are testers on the team, it is obviously the developers’ job to give them things to test, and the testers’ job to tell the developers if their code works.