Top 5 Criteria for Measuring QA Team Success

Quality Assurance teams think of software developers as individuals who drink highly caffeinated beverages while conducting frantic bug hunts under tight deadlines. In reality, the truth is much different.  

QA teams work as the support backbone for the development team, and the best ones drive the quality of the product through thorough process-driven functions. The value these teams provide often goes unnoticed, but they are at the foundation for driving operational agility and meaningful customer interactions. 

So how do we measure success? Is it about how many software bugs a team reports or about the quality of those bug fixes?  

Not too long ago, the number of bugs in your database was the primary indicator of your game’s quality. As games have become more and more complex, so have the criteria to analyze and measure quality. It is also important to note that time and effort (and to be frank, cost) of testing games has risen accordingly. 

So, with all this in mind, we’ve put together a list of the top 5 data-driven criteria that any QA manager needs to know to keep their team operating at optimum velocity.  

1. Bug Acceptance Rate

This is a  great barometer to track QA team reporting. It will showcase valid bugs and reduce duplicated and invalid ones. It provides an excellent high-level overview and synopsis of how your QA team is doing. The Bug Acceptance Rate is a one-stop KPI that is open for both the QA management and the Development team to view how the bugs are being reported. It shows the QA team understands the product and the development priorities.

2. Qualitative Bugs

It is not always about the quantity of bugs; however, it is primarily about the quality (important and critical) of bugs. What percentage of bugs matches your severity criteria? How many reported bugs are there on stop-shipment issues? A great QA team will find critical issues to the usability of the product and match against development priorities. This stops the dev pipeline being clogged up with low severity issues. 

3. Task Productivity

This shows the time the QA team takes to complete the assigned tasks, such as regressions, and test cases. With good test planning, the QA team clearly understands what to do and when to do it by. This is easily trackable and can be used to hone productivity over time.

4. Task Accuracy

It is essential to complete the tasks accurately. When testers are familiar with the tasks, they tend to mark it as a pass without thoroughly checking it and mismarking them. This should not happen; a QA team should always perform such tasks diligently. A great QA team will have processes to help each other be diligent in task completion, including regular peer-review and management oversight. This extra oversight is easily trackable and provides a direct report on task accuracy. 

5. Fix Failed Bugs

This helps the development team understand how sticky their fixes are. The result is a quicker time to market for game developers and publishers. As someone once said, behind every great developer, there is a great QA team, and nothing shows this off better than a low fix-fail rate. It is the true representation of a QA and Development team working in tandem to create a great product. 

If you have any questions or wish to understand how GlobalStep can use these KPI’s to help your Development teams produce better product, get in touch with us at: sales@globalstep.com






READ MORE


The What, Why and When of Usability Testing

We all know that a reduced or sub-par end-user experience will impact your bottom-line. But how can you ensure your products work as intended and can provide a high level of confidence your customers will love? 

Confidence in your digital products will remain high when you engage with Digital Assurance services, and we’d like to discuss one aspect of that: Usability Testing. 

Software development teams are often the unsung heroes who work tirelessly to make apps people will love. You don’t need to go at it alone, though. You need a partner that can provide the peace of mind you deserve. This is where GlobalStep can assist. 

Below are some key considerations for Usability Testing. Keep in mind that Usability Testing is part of any broader Digital Assurance initiative responsible for ensuring your confidence in your digital products. 

What is Usability Testing? 

Usability testing is traditionally a method of testing an app, website, game or software’s functionality and observing how users interact with the product. More specifically, it helps identify the “goals” a user will want to achieve within the product. 

Recording how users interact with a product and moving through the product to achieve the desired goals ranges from written statements to in-person observations. 

Essentially, testing will be focused on specific goals or across the entire product. Before starting the test, our team will work with you to create a clear plan to cover all aspects that need testing, and it will deliver the answers to your goals.  

This can range from looking at an onboarding process (Goal: Do our end-users enjoy a seamless and faultless on-onboarding flow?) or it could be the First Time User Experience (Question: Do our users understand how to use the product after using the product for the first time?).  

It might also mean executing an ad-hoc test across the entire product (Question: I need to know if the product works as an end-user expects it to) or providing feedback on ease of use (Question: I need to know if the end-users find the product friendly to use). 

As our usability testing uses trained professional testers, we will also pick up functionality bugs and will test against test cases to ensure the product also works as you expect it to. 

Why do you need Usability Testing? 

How do you know what experience your end-users have when they pick up your product? Best practice means you will already have analytics built into the product. Analytics does a great job of visualizing aspects like where users travel, whether they drop-off at specific points, and what areas are “sticky”, among others.  

The problem is that the data is anonymous and leaves you guessing why a user followed that path, quit the app at that point or why they spent all their time in one area. 

Usability testing will reveal those answers… and many more. 

Our usability test reports will allow you to understand why end-users took specific actions or behaved in a certain way on their journey to achieve a particular goal. Armed with that information, you can tweak, amend or even pivot the design to relieve the specific pain point. 

GlobalStep’s experience

Our Usability testers have a strong pedigree of identifying facets of the user experience leveraging expertise from testing in the Interactive Entertainment industry. Identifying areas that hamper the quality of the end-user journey is a regular feature of our testing.  

GlobalStep aims to leverage this experience for Usability Testing for apps and softwares across multiple domains. Best practices gathered through numerous engagements are also leveraged to provide excellence in usability testing. 

Usability testers can also contribute to finding functionality bugs in their testing. In fact, these functionality bugs might also be an unseen factor that you haven’t spotted in the user flow. Bugs are reported daily by our teams so you will always be able to react quickly. 

Alongside the usability testing, we can also run specific functionality test cases. Thus, besides understanding how your end-users want to operate the product, you can rest assured that it also works in a manner specified by you. 

When to use Usability Testing? 

There are quite a few times through the development cycle where usability testing is fundamental. Testing should be scheduled multiple times through the development of a product to ensure you always have a great understanding of how end-users are interacting with your product. 

Prototyping 

  • – Find early-stage issues and stop them from being baked in through early stages of product development. 
     

Feature Development

  • – As new features are developed, pass them through to the usability and functionality test team to test and ensure that they work as required. 
  • – Make sure that new features fit within any design constraints or premises. 
  • – Don’t leave testing until all the features are completed! 
     

Soft launch or just before the main release

  • – As your product gains more traction in the marketplace, it will attract more users. 
  • – Pay particular attention to FTUE and any purchasing points – these are going to get a lot more attention now! 

Post-release

  • – Gone are the days when we would release a product and let it live or die from the first release. 
  • – Regular updates to add features, fix bugs or improve the end-user experience will need validation – both from the functionality and usability perspective. 

Hopefully, this should provide some guidance on how to get started on the journey to increase your confidence in your digital products. In the meantime, if you have any questions or wish to speak with a GlobalStep QA expert, please get in touch with us. 

Get in Touch:






READ MORE


5 Benefits of Outsourcing Game Testing and QA Engineering

Is outsourcing tasks within an organization worth it? Let’s break it down. 

On one hand, outsourcing tasks to an experienced partner can help drive cost savings of 20% to 40% depending upon process complexity and capabilities of the service provider. On the other hand, there’s always a nagging doubt in the mind of the management regarding the ability of the service provider in understanding the typical complexities of the market, aligning to their studio culture and delivering on promised service level agreements.  

While both approaches have their pros and cons, increased process orientation, better project management and the demand-supply gap for experienced testers are making the case for outsourcing stronger. The current push to work remotely as warranted by the COVID-19 pandemic has also made studios more receptive to outsourcing than before. 

Software Testing traditionally has had a rich history of outsourcing success. While Game Testing is conceptually similar to software testing, it has its own nuances and challenges. Game Development studios are constantly struggling to make the right decision between in sourcing and outsourcing. But with a market that’s growing fast both in terms of the number of gamers as well as the number of devices games are being played on, the scope of game development and testing has amplified manifold. With the advent of new technologies and platforms such as Stadia, Apple Arcade etc. it has added on to the complexities of testing to provide optimum coverage in the shortest possible time. 

Global Games Market Forecast (Newzoo)

Game developers and publishers now need to ensure a much wider testing coverage to maintain the same levels of quality as doled out before. Agility of testing also needs to improve to test the multiple updates and patches that developers are now rolling out to gamers within shortened schedules. This is where an independent game testing company can help developers get access to the right talent at a cost better than companies can manage internally. Today, we look at how independent game testing service providers like GlobalStep can help your business. 

Why should you choose to outsource?

  • Gain access to an experienced team of the industry’s best game testers:

Game testing service providers will provide you access to the best-skilled game testers. These teams usually have an experience of working in different genres of games and on multiple platforms. Their credibility, experience, and a wide perspective ensure that defects are identified and communicated at an early stage, enabling you to maintain high-quality standards. 

  • Ensure complete coverage with the best tools:

Experience with best-in-class tools, technologies and frameworks can be a key factor to ensure that your game is tested inside out. Game testing tools and technologies such as Appium, Robotium, Calabash, Device Anywhere typically require a different skillset.  

An independent testing service provider can help you gain access to the best tools in the market as well as the resources trained to use these tools to maximum advantage. Advanced project management methodologies and proven frameworks can further enable independent service providers test various scenarios faster and more effectively than you can internally. 

  • Optimization of up-front and ongoing investments

All the testing needs discussed above are time- and resource-intensive activities. Sometimes, production houses also put the onus of testing on developers – which is not an ideal use of their time. An experienced game testing provider can provide access to the best resources, tools, platforms, and devices at a fraction of the cost, allowing you to concentrate on the core processes of your organization and expel your resources on your business priorities.

  • Achieve a faster go-to-market

Even a small bug can bog down a game’s experience a great deal. With the internet providing a voice to consumers, a negative comment can quickly evolve into a situation which can hamper the success of an entire game.  

However, as comprehensive and competent as your developers may be, bugs are inevitable. The only solution is to test effectively, find the bugs and fix them. Unfortunately, this is time-consuming and with the fast pace of the market, simply not possible anymore!  

Automation tools and frameworks available with experienced game testing partners can help reduce the time and manual effort needed in testing. With the advent and implementation of new transformation methods – using telemetry for reducing test efforts, risk-based testing, shift left, and other means, you can further reduce your time to reach the market. Test service providers can also leverage experience from working with various customers and implement best practices from across the industry for your project. This enables you to get your game as well as patches to the market faster, giving you a huge competitive edge! 

With the dynamics of the modern gaming industry changing at a very fast pace, game development studios must consider a change in their testing strategy. Increased usage of tools, automation, data analytics and collaboration between developers and testers is going to be the key to rising above competition and outsourcing game testing can help you achieve all of these!  

It is, however, also key to evaluate service providers on key criteria such as experience, project management frameworks, quality of their QA labs and testing infrastructure, as well as adherence to metrics and SLAs. We wish you luck in your journey towards finding the right partner! 

[Rachit Jain is Sr. Manager – Solutions Engineering at GlobalStep. He is a management professional with a technology background who loves to analyze and understand business needs and devise the integrated end to end solutions that fits the requirements. At GlobalStep he works closely with CXO’s and senior management to devise strategies for business development and devise a game plan to achieve the set goals.]

READ MORE