Your guide to the opportunities, emerging trends, challenges, and solutions that are shaping the future of the game quality craft
Estimated read time: 6 minutes
As the global game quality community gathered for Qualicon 2024 this May, the focus was on the themes of ‘innovate, advocate and add value’.
Across two-days of talks, knowledge-sharing, and discussion, together that trio of topics provided a framework for a diverse and future-facing schedule covering everything from the impact of emerging technologies on the craft, to strategies for asserting the value of implementing the game quality disciplines as early as possible in game production pipelines.
There was, however, one further theme that underpinned almost every moment of Qualicon 2024, and in the best way possible. The sense of community that permeated the conference was so rich in energy, enthusiasm, and excitement about its own collective potential, it was impossible to miss. Anyone who works in the quality field will have very likely come across persistent misconceptions about its reality; perhaps that QA is primarily a gateway role to other game dev specialties, or notions that testing frames the entire skillset. And yet at Qualicon, with hundreds of quality professionals coming together to elevate the potential of their work, the optimism about what was possible together – advocating, innovating, and bringing value to games making – offered more than potential. It felt absolutely a starting point for meaningful change.
The audience were as much a part of that effort as Qualicon’s speakers, filling breakout spaces with impact-focused conversations, and becoming parts of panels and talks with their questions and insights.
But what were the key discussion points and learnings from the day? If you’re a Game Quality member, you can catch up with all of the Qualicon content over in the Vault. For those looking for an overview, however, we’ve rounded up those that permeated so many Qualicon panels, talks and roundtables right here:
1. GAME QUALITY SHOULD INSPIRE ORGANISATIONAL CHANGE
Monitoring and addressing the quality and function of a studio’s games is absolutely a core focus of game quality teams and individuals. At the same time, those of us in quality roles – and particularly leaders – can also do much to inspire cultural shifts across studios that can adapt how a game company considers ‘quality’, and therefore the impact and potential of game quality teams’ work. If we can communicate our work using constructive, open language with kindness and empathy, encouraging dev teams to define themselves by their successes while embracing the learning potential of failures, all the while recognising that having work ‘tested’ for quality can expose vulnerability and trigger insecurities, a change can be introduced that goes beyond an individual title.
“A little bit of this language can go a long, long way in bringing our teams, bringing our organisations, and bringing what it means to do ‘quality work’ into a new space,” proposed Nate Stoddard, QA Leader and Advocate, during his talk on game quality leadership. “If we as quality champions are able to address how a studio may be defining itself by its failures, rather than by its success, that is a different measure of the value that we’re adding as quality champions.”
Ultimately, this effort can bring about a quality of organisational culture that serves as a foundation for better work across a studio, while ensuring a better relationship and communication between game quality professionals and their peers. Stoddard recommends that all who work in game quality read the book Dare to Lead by Brené Brown. Because all of us who are not leaders by role are still part of a movement that can advocate our potential to studios and elevate the value and impact of our collective work.
2. LIVE SUCCESS STARTS WITH END GOALS
The rise of live, maintained, persistently updated games as a new standard of player expectation has dramatically shifted the work of game quality. Beyond live games consistently introducing new content and systems (that must behave well with all existing content and systems), they also exist in highly dynamic, always-online forms. They boast an infinite capacity for change, and a very real example of a product that can never be fully, completely tested. That presents obvious challenges for the application of game quality processes.
“It starts by understanding what the goals are. That means the business goals, and also the product goals. You have to have a good understanding of what your limitations are with your team,” suggested Chris Ime Etuk, Senior QA Manager at Epic Games, during a panel on live games. “Having that knowledge [of your goals] you’re able to – in a perfect world – set and manage expectations accordingly, on what your team can deliver, and what they cannot deliver. Really at the root of this is your test plan, which is your primary communications source for you to communicate these things out.”
Reverse engineering that test plan from the goals and realistic limitations, then, is key to maintaining quality in live games. That test plan will need the ability to adapt or evolve, but essentially, establishing it early and communicating it clearly to the wider dev team in the opening stages of a game’s production is key; as is honesty that not everything is possible when it comes to live games. Careful selection of priorities here matters much more than stretching ambition beyond reality. Better tooling and automation offer some of the most meaningful solutions to this challenge, but they too should be informed and guided by those same clear end goals.
3. MANAGING PLAYER COMMUNITIES AS QUALITY TEAMS
Players today commonly expect to be relatively involved in the development and evolution of a game. That is in part because notions like early access and developer diaries are new standards of game marketing and engagement, but also because live titles necessarily introduce a production loop where player activity informs the likes of future updates. It was put forward several times across Qualicon 2024 that players now expect to provide input – and have their feedback listened too. That presents a challenge for game quality; namely how to best extract and interpret meaningful and implementable insights from vast pools of player feedback data. At Qualicon 2024 considerable time was devoted to discussion of the need to build more simple, intuitive player-facing, even game-specific tools that ensure players are heard, while steering them to points where particular issues require player feedback. These tools should also guide and funnel player feedback on those areas so that it is delivered in actionable ways, keeping their focus on the elements of game design and production they can meaningfully understand.
“An ‘external dash[board]’ public database is something that really worked well with us,” offered Bruno Senecal, QC Project Manager at Ubisoft Montréal. “It kind of offered a way to have this discussion with the community; a place where we could acknowledge issues, […] and where we could grab more information is a good way to start. Having active moderators on social [media] as well – again, just to build those bridges. Those moderators can – in combination with QA – work with [those players].
It was also noted that high profile community members and on-side content creators can be a voice both to manage player expectation around feedback, and guide insights.
4. RESISTING ‘THE THEATRE OF INNOVATION’
Innovation presents a simple concept on the surface; it is about the pursuit of new ideas. However, truly valuable innovation should really be guided by unearthing problems and seeking solutions. That is not to say all innovation in game quality should only be about tangible impacts. Innovation can be exploratory. Innovations that fail can still deliver key learnings. But innovation should never be practised for its own sake, or to assert being innovative in pursuit of seeming so. Those acts are not truly innovative but – rather – the ‘theatre of innovation’.
“Innovation for me is about: ‘Where do we see that the problems are? Are there unmet needs? Where do we need to improve efficiency? Is effectiveness where we need it to be?’,” stated Adrian Brown, Director, Data and AI, at Studios Quality within Xbox Game Studios, pondering what should guide a game quality team’s motivations. “Innovations at their core are new ideas and methods. They’re new solutions – but new solutions to what? We don’t want to innovate because it is cool. We want to innovate to solve a problem. When you’re looking specifically at that problem – that unmet need – then you start to see ‘here are the metrics that we need to measure that against’.”
Measuring innovation through metrics should come naturally to game quality professionals. And remember, when trying to innovate in the game quality crafts, simplicity and elegance are among the most critical of those metrics. Innovations that add complexity tend to increase capacity for error or miscommunication. Innovate for need, and innovate simple, and your game quality efforts have a much higher chance of greater impact.
5. INNOVATION – AND GAME QUALITY – THRIVES IN SAFE SPACES
Many innovations in game quality are about pursuing or enacting cultural shifts. That’s even true when developing a new tool, methodology or procedure, in that such things may require such a shift so they are better understood or embraced. Innovation is also considerably more likely to succeed in a culture where failure is welcomed as a tool for learning, and where all manner of seniority levels and disciplines are given the opportunity to have input and be heard. Innovation efforts – be they a two day on-site event or a long term tool development procedure – should be inviting, celebratory, empowering, and even playful. All those things together create a safe space for innovation. In fact, it was said that those things are also key tenets for the practice of the game quality craft generally; not least because the work of game quality bears much comparison with the process of innovation.
Almost all innovations in our realm should also acutely consider that game quality is a speciality that serves, supports, and empowers game creation across its production and teams. As such, bringing those from outside game quality’s disciplines into our innovation projects and procedures can deliver end results and solutions that are considerably more relevant and impactful. Equally, external voices can provide refreshing or unexpected perspectives.
“We need to advocate to empower our employees from different departments to collaborate, with our department, and outside our departments also,” Ritu Chowdhary, Head of QC and Test Automation, Ubisoft India Studios encouraged the Qualicon 2024 audience. “Stay curious about each other’s work, and understand the bigger picture, so that we are nurturing creativity and experimentation [across games]”
6. STANDARDISATION IN THE MODERN ERA (MOSTLY) LIBERATES GAME QUALITY’S POTENTIAL
Whether working with an external QA partner, or providing third party support to a studio, or collaborating within a remote or hybrid game developer, working across partners, time zones, locations, and more is common now than ever before. This presents many challenges, from information security to bridging and coordinating contrasting working practices. Those factors, along with the practical realities of operating across multiple locations or organisations can slow or complicate the work of game quality.
The solutions here are multifaceted and widely varied, but through Qualicon a mindset was discussed that provides a simple and universal way to think about the conundrums of working with partners or across sites – an increasing reality of the modern post-Pandemic era.
In all those cases establishing a common language and vernacular, along with standards across test plans, frameworks, and quality metrics can be tremendously helpful. Those things benefit clarity and communication, preempting missteps, avoiding major errors born from minor miscommunications or misassumptions, and supercharging efficiency.
Standardisation can equally potentially stifle the flexibility demanded by individual projects, and efforts in the innovation realm. Claiming the benefits of standardisation without inadvertently adopting the limitations can prove a delicate process. It was positioned at Qualicon 2024 that the solution may simply be a form of flexible standardisation; that is, carefully and meticulously adopting standards on a per-project or per-partner basis – making sure to always thoroughly communicate the new standards (and any that remains unchanged, because clarity wins out in these cases).
UNTIL NEXT YEAR!
There were many more talking points and fascinating conversations at Qualicon 2024, of course. And again, you can dive into all the recorded content from the conference over at the member Vault. We’ll be back next year with Qualicon 2025, and until then keep an eye on our events page to stay in the loop with all kinds of other game quality events, panels, online gatherings and more.
And if you were there, on stage or in the audience, a sincere thank you for being part of it. The energy, community, enthusiasm, knowledge sharing, experience, innovation, advocacy, and support on show at Qualicon was a reminder of why we set up Game Quality.
