The GoalThe goal of the Critical Defect Review Process is to understand the root cause of why any high-priority defect exists in production and take specific remediation actions to ensure that this issue and others do not occur again. We also establish ownership of any actions going forward. Finally, we keep a recorded log of these issues, which will be reviewed periodically. Note, that the goal is NOT a "witch hunt" in terms of assigning blame. If we are not open to deep understanding, we will never be able to solve the problem. The ultimate goal of this process is to deprecate the process when we have zero critical defects in production, however this occurs.
We continue to monitor this process and continuously inspect and adapt it, but currently, this is the basic framework of our Critical Defect Review Process.
PeriodictyThis review occurs every three weeks (the same cadence as our sprints) with all resolved production defects marked as either a "Priority 1" or "Priority 2." The reason we wait until they are fixed is that the highest priority is to resolve any customer-critical issues, regardless of their genesis. Additionally, reflection generally helps towards the further understanding of the root cause of problems, and the eventual solution.
ParticipantsWe have the following roles participate at these reviews:
- Moderator/Scribe (generally a member of the Development Management team)
- Product Technical Lead
- Primary Engineer involved
- Primary QA Engineer involved
- QA Lead
- Customer Support Lead
- All team members are, of course, welcome to attend and listen
FormatThe discussion captures the issue in question, any related issues, the primary engineer associated with the issue, the primary QA engineer associated with the issue, the root cause/synopsis, recommended actions moving forward (remediation), and agreed actions. All of this is captured in our wiki. Note, the agreed actions may be an acceptance of a certain risk within our process, although this is not the primary desired objective. This information will be reviewed at a regular basis by Product Development Management, Executive Leadership, and the team, at the very least during the sprint retrospective.
Again, there is no blame assigned, which can only work within the larger context of "The Socialware Way" where our team members are respected, trusted, and treated as our most precious assets as opposed to "resources."
While we have only been doing this for about two months, we've seen some fairly impressive results. Not only are the issues trending down, we are also able to respond more quickly and have a deeper understanding of why quality problems exist. Of course, we are implementing a number of other changes to our product development system that result in high-quality customer value as a natural outcome of our system rather than something that needs to be forced. So, the attribution of the increased quality is manifold. I will continue to monitor and share results.
The genesis of this concept comes from the concept of a Quality Circle, which was introduced by Dr. Ishikawa. It was interpreted into "cross-functional teams" which was a key part of some Total Quality Management systems.
I would love to hear your thoughts on this process, please feel free to direct message, or comment below. Thank you for your time in reading!