Take Blizzard for example. They just released a new patch, where class campaign quests for 8/12 classes do not work. Sure, it’s a remixed version of older expansion, and with all the phasing stuff I can kind of imagine some of the phasing issues being caused by, I don’t know, the player having a weird combination of completed stuff that’s hard to properly catch in testing, since there’s quite a lot of variables.
But the fact that one of the class quests requires crafted items to be completed, while crafting isn’t available by design in the Remix, there’s just no excuse. They either just don’t give a fuck about an issue that’s literally a progression blocker with 100% repro rate (while also being pretty easy to fix), or no one ever tested it even once. And it’s not just some random sidequest, it’s literally the main class campaign, one of the main features of the expansion.
As someone who worked in QA and gamedev, I can’t imagine how could something as obvious as this ever get approved for release. That’s something you catch immediately. Hell, you don’t even have to play through it to realize that this might be a problem.
Work at a larger company. Most people are so used to terrible Customer Service these days that we just use our customers as the QA. Nobody complains as much as they should. As they say
Everybody has a test environment. Only some are lucky enough to have a separate production environment.
From a tech POV, that makes a lot of sense. Use AI to find the needle in the haystack. Then let a person validate. That’s probably one of the better uses for it. Although I don’t love AI for any of the broad reasons to not like AI.
Maybe before. But it’s gotten pretty damn good at detecting anomalies and issues. And every time a human QA validates the info, it gets better.
I’d still leave it to a human to fix the code though. I suspect that letting AI write the code would make it unworkable for people in the future. But maybe it can write code in a straightforward way to be managed. I don’t know. It’s advancing pretty fast.
It has nothing to do with AI, it’s just plain old automation, but they solve most of the issues you get with making automated tests in non-discrete 3D playspace and they do that in a pretty solid way. It’s definitely something I’d love to have implemented in the games I’m working on, as someone who worked in QA and now works in development. Being able to have mostly reliable way how to smoke-test levels for basic gameplay without having to torture QA to run the test-case again is good, and allows QA to focus on something else - but the tools also need oversight, so it’s not really a job lost. In summary - I think the talk is cool tech and worth the watch.
However, I don’t think AI will help in this regard, and something as unreliable and random as AI models are not a good fit for this job. You want to have deterministic testcases that you can quanitfy, and if something doesn’t match have an actual human to look at why. AI also probably won’t be able to find clever corner-cases and bugs that need human ingenuity.
Fuck AI, I kind of hope this is just a marketing talk and they are actually just improving the (deterministic) tools they already have (which actually are AI by definition, since they also do level exploration on top of recorded inputs), and they are calling it an “AI” to satisfy investors/management without actually slapping a glorified chat-bot into the tech for no reason.
Frankly, this is good news. Whoever buys the rights to kingdom hearts in 3 years when the company falls apart might manage to create an intelligible storyline.
Kingdom hearts 1 was a coming of age story with some fantastic elements tossed in
Kingdom hearts 2 was about antipathy and how it destroys the world, with some, uhh, who was that guy? And why’s the bad guy on my side?
Kingdom hearts 3 was, wait, why was he cloned? When was he cloned? When was she, and him, and him again? And his third clone was a girl? And whose heart was imbued into what? What war? What? Who? What???
I get that getting all the games as they released was hard, because the series is on so many platforms. But I really don’t get the “KH is hard to understand” argument today, because you can easily find hundreds of letsplays for every game, cutscenes complications, play/watch every game on the PS4 remix disk, and even watch a fandub of the mobile games (Dark Road is a WIP) if you don’t like the KHUX Back Cover recap.
So like, what’s so hard? If you skip games and only read a wiki (the worst possible way to consume any sort of media, mind you), of course you’re not gonna know the story and characters, and of course it’ll sound confusing.
Dude, it’s multi-author-comic level bad. I’ve skipped entire sagas in several book series due to a lack of translations and ended up less confused. It’s green arrow levels of clone shenanigans.
To be clear, I’ve played most of the games and they’re still ridiculously difficult to keep track of. All besides the mobile, early non-Ventus card mechanic arpg, and the disappearing girl clone sora game.
They’d be easier to follow if they stuck to the rules of their own universe. Body and heart separate and the body persists not once but twice? What?
If you hate Ubisoft because they put a black person in their game, and not because they are greedy bastards who make utterly vapid slop, then you are an idiot.
Well, it’s not game development, but bugfixes and quality testing.
I dont know, but it does makes sense, when there’s still 30% work being done by human eyes. There will still be people checking everything through.
Even if they hit 50-50, they could put more money into the development.
The argument that they will just save the money only works as long as another company doesnt use it for game devs. Otherwise you naturally fall behind.
It also only works as long as the AI can actually competently do the QA work. This is what an AI thinks a video game is. To do QA, it will have to know that something is wrong, flag it, and be able to tell when it’s fixed. The most likely situation I can foresee is that it creates even more work for the remaining humans to do when they’re already operating at a deficit.
To be fair, that’s what an AI video generator thinks an FPS is. That’s not the same thing as AI-assisted coding. Though it’s still hilarious! “Press F to pay respects” 🤣
For reference, using AI to automate your QA isn’t a bad idea. There’s a bunch of ways to handle such things but one of the more interesting ones is to pit AIs against each other. Not in the game, but in their reports… You tell AI to perform some action and generate a report about it while telling another AI to be extremely skeptical about the first AI’s reports and to reject anything that doesn’t meet some minimum standard.
That’s what they’re doing over at Anthropic (internally) with Claude Code QA tasks and it’s super fascinating! Heard them talk about that setup on a podcast recently and it kinda blew my mind… They have more than just two “Claudes” pitted against each other too: In the example they talked about, they had four: One generating PRs, another reviewing/running tests, another one checking the work of the testing Claude, and finally a Claude setup to perform critical security reviews of the final PRs.
I don’t know what they were testing, but if your output is text, it will be a lot easier for the AI to know it’s correct than any of the plethora of ways that video games can go subtly wrong, and that’s where my lack of faith comes from. Even scraping text from the internet, my experience is more often that AI is confident in its wrong answer than it is helpful.
videogameschronicle.com
Aktywne