In the last post in this series I discussed the process involved around the first part of the ideal bug process. In this post I will finish up the rest of the process and metrics for handling bugs.
This state indicates the bug has been fixed according to the developer.
This means that the developer has fixed the code, checked it in and associated it with the appropriate bug work item. They have run the test case provided by the analyst in the Under Investigation state. In addition, the time between the bug being in the Active state and the Fixed, unverified state is the time it took the developer to fix the bug. In other words, we now have an approximation of the developer effort to fix the bug. It doesn’t provide an exact time but it provides an overall throughput time.
Ready for Testing
This state indicates the bug has been assigned to a tester but the testing has not yet started.
There are a number of reasons for this state. The first is that your branching model is such that a tester does not have access immediately after the code has been finished (i.e. you are waiting to promote code to the QA branch). The second is that there is simply other work going on and that the tester just can’t get to it right now. The transition between Fixed, unverified and Ready for Testing should be made by a Test Manager to acknowledge that the bug has been assigned to a tester (or to the testing group in general) and that it is on the right code branch for testing. The transition to this state does not provide any other meaningful information.
This state indicates that the bug is actively being tested.
The other important information provided by this state is the amount of time between the bug being ready for testing and when the testing started. This again shows waste.
This state indicates that the bug has been fixed according to independent testers.
The time between this state and the In Testing state provides the amount of time spent testing this bug and is considered work time.
Note: For those familiar with Lean theory, this work would be classified as waste. The reason is because work should never pass a station with defects – it means the process for building the code needs to be fixed. However, as I have mentioned before, I believe this is mostly impractical in software development.
Scheduled for Deployment
This state simply indicates that the fix is scheduled to be released.
An important piece of information to capture at this state is the release that the fix is being deployed in. This helps create a Release Notes document to provide the user explaining the changes in the upcoming release.
The bug is closed after the release that it is included in has been deployed.
The time between the bug report being Proposed and the bug being Closed provides the entire throughput for the bug being fixed. This information is a good metric of whether or not your process is improving (if reducing the amount of time to accept and fix bugs is one of your goals). While removing individual pieces of waste is important, you have to keep your eye on the overall goal.
A note about UAT
You may be wondering about the lack of inclusion of User Acceptance testing steps to track users’ acceptance of the bug. There are two reasons why this isn’t included. The first is, does it provide you any additional information? Sure, it lets you know the users accepted it, but you can do that as part of the QA process. Second, by doing UAT separately it means that you have to have separate UAT environments from QA which many organizations can’t afford. Is the cost worth it? This may be something you need to take into account as you think through your process.
This finished the process and description of each step in the process. The next post in this series will cover Roles & Responsibilities and Reports and how to track improvements in the process.