Top 5 Criteria for Measuring QA Team Success
Quality Assurance (QA) teams conjure up images of software developers drinking highly caffeinated beverages conducting frantic bug hunts under tight deadlines. In reality, the truth is much different. QA teams work as the support backbone for the development team, and the best QA teams drive the quality of the product through thorough process-driven functions. The value these teams provide often goes unnoticed, but they are at the foundation for driving operational agility and meaningful customer interactions.
So how do we measure success? Is it about how many software bugs a team reports or about the quality of those bug fixes? Not too long ago, the number of bugs in your database was the primary driver to show your game’s quality. As games have become more and more complex, so have the criteria to analyse and measure quality. It is also important to note that time and effort (and to be frank, cost) of testing games has risen accordingly.
So, with all this in mind, I’ve put together my list of the top 5 data-driven criteria that any QA manager needs to know to keep their team operating at optimum velocity. They are;
1. Bug Acceptance Rate: A great barometer to track QA team reporting. It will showcase valid bugs, reduce duplicated bugs and invalid bugs. It provides an excellent high-level overview and synopsis of how your QA team is doing. The Bug Acceptance Rate is a one-stop KPI that is open for both the QA management and the Development team to view how good the bugs are being reported quickly. It shows the QA team understand the product and the development priorities.
2. Qualitative Bugs: It is not always about bug quantity; however, it is primarily about the quality (important and critical) bugs. What percentage of bugs match your severity criteria? How many bugs report on stop-shipment issues? A great QA team will find critical issues to the usability of the product and match against development priorities. This stops the dev pipeline being clogged up with low severity issues.
3. Task Productivity: Time taken to complete the assigned tasks such as regressions completed, test cases covered, tasked covered, all reviewed in total hours. With good test planning, the QA team clearly understands what to do and when to do it by. This is easily trackable and can be used to hone productivity over time.
4. Task Accuracy: It is essential to complete the tasks accurately. When testers are familiar with the tasks, they tend to mark it as a pass without thoroughly checking it and mismarking them. This should not happen and always should perform such tasks diligently. A great QA team will have processes to help each other be diligent in task completion, including regular peer-review and management oversight. This extra oversight is easily trackable and provides a direct report on task accuracy.
5. Fix Failed Bugs: This helps the development team understand how sticky their fixes are. The result is a quicker time to market for game developers and publishers. As I always say, behind every great developer, there is a great QA team, and nothing shows this off better than a low fix-fail rate. It is the true representation of a QA and Development team working in tandem to create a great product.
These are just a few tips that I’ve learned over the years. I’ll be posting more in the weeks to come. In the meantime, if you have any questions or wish to understand how GlobalStep can use these KPI’s to help your Development teams produce better product, please email me directly at Ben.Gunstone@GlobalStep.com.
Get in Touch: