A lot of hate in the comments but IMO this is one of the few things that LLMs are actually really good for. It’s a shit job nobody wants to do that LLMs are really good at. Notice that they said 70% and not 100%. Yeah that means they’re probably going to have 30 people doing the work that 100 people used to do but people are still in the picture overseeing things. Automation isn’t, by itself, bad. The bad part is that our whole society is built on the idea that your entire value as a person is based on being able to work and make money and job loss is way worse than it should be.
As a developer, it bothers me that my code is being used to train AI that Square Enix is using while trying to deny anyone else the ability to use their work
I could go either way on whether or not AI should be able to train on available data, but no one should get to have it both ways
So… im a big supporter of squeenix, buy everything they make… but this tells me the quality of their games is going to go down the toilet. Knowing AI it’ll come up with fake lists of bugs that didn’t happen and all the real bugs will not be listed and they’ll release the buggies shit. One thing I LIKE about square, being one of the few companies I do pre-orders from still, is that their products are fairly bug free on launch FFXVI had some graphics optimization issues, but I’ve been happy with most of what I got the past few years.
Cool, but I wish they’d look at why people hate them, ignore the bullshit reasons, and focus on the good points — and work to improve them.
I haven’t cared about Assassin’s Creed since the first one. The fights are like playing Guitar Hero blindfolded, but on Expert, except if you fail, you die and have to start over. QTEs where you can’t even see the prompts are dumb. They had a cool vision for the game, but the actual implementation sucked.
Another game, I forget which one, had forced inverted X-axis and you couldn’t un-invert it. Bonus, the Y axis couldn’t be inverted like I like. So the game was completely unplayable.
Ubisoft is a weird beast. They keep putting out the same slop time after time and people keep buying it up. I think Ubi is just really good at production quality and marketing, and this is apparently good enough to garner sales over and over again. It’s like…they have talented devs, but they don’t let them cook. They have a very “top-down” approach to design to encourage consistency in their catalog, but it seems like it’s also what has caused them to stagnate.
On the one hand, I appreciate that they are fighting this fight and trying to put a kibosh on the next gen of gamergaters. On the other hand, I kind of don’t care so long as every game they put out is all flash and no substance.
Yeah none of those reasons were why people hated Shadows though. It was entirely racists. They aren’t gonna make any more improvements to their formula because Ubisoft doesn’t innovate, they’ll just wait for someone else to add an interesting twist to the open world formula and then steal that.
Oh yeah, I heard people didn’t like the idea of a Black guy in the Japanese AC game. Is that the one Shadows was? So yeah, that person may not have existed in history. Then again, neither did Robin Hood. Stories don’t need to be based in fact, and as much as the original Assassin’s Creed was all about historical accuracy, the whole Animus thing placed the game entirely within the realm of fiction and fantasy, which tells me any artistic liberties they take with the history you dive into is A-OK in my book! So I don’t care if that guy existed IRL or not. I was intrigued by the idea of a Japanese AC game… but put off by the developer.
And racists are idiots, just, full stop on that count.
he also 100% existed lol. Yasuke has tons of historical records about him because he didn’t exist in like ancient Japan, he was the bodyguard of Oda Nobunaga during the Sengoku era. like you said, racists are just idiots and can’t stand actual history that goes against their stupidity.
Exactly, as I don't expect QA done by something that can't think or feel to know what actually needs to be fixed. AI is a hallucination engine that just agrees rather than points out issues, in some cases it might call attention to non-issues and let critical bugs slip by. The ethical issues are still significant and play into the reason why I would refuse to buy any more Square Enix games going forward. I don't trust them to walk this back, they are high on the AI lie. Human made games with humans handling the QA are the only games that I want.
Exactly, as I don’t expect QA done by something that can’t think or feel to know what actually needs to be fixed
That is a very small part of QA’s responsibility. Mostly it is about testing and identifying bugs that get triaged by management. The person running the tests is NOT responsible for deciding what can and can’t ship.
And, in that regard… this is actually a REALLY good use of “AI” (not so much generative). Imagine something like the old “A star algorithm plays mario” where it is about finding different paths to accomplish the same goal (e.g. a quest) and immediately having a lot of exactly what steps led to the anomaly for the purposes of building a reproducer.
Which actually DOES feel like a really good use case… at the cost of massive computational costs (so… “AI”).
That said: it also has all of the usual labor implications. But from a purely technical “make the best games” standpoint? Managers overseeing a rack that is running through the games 24/7 for bugs that they can then review and prioritize seems like a REALLY good move.
They're already not paying for QA, so if anything this would be a net increase in resources allocated just to bring the machines onboard to do the task
I would initially tap the breaks on this, if for no other reason than “AI doing Q&A” reads more like corporate buzzwords than material policy. Big software developers should already have much of their Q&A automated, at least at the base layer. Further automating Q&A is generally a better business practice, as it helps catch more bugs in the Dev/Test cycle sooner.
Then consider that Q&A work by end users is historically a miserable and soul-sucking job. Converting those roles to debuggers and active devs does a lot for both the business and the workforce. When compared to “AI is doing the art” this is night-and-day, the very definition of the “Getting rid of the jobs people hate so they can do the work they love” that AI was supposed to deliver.
Finally, I’m forced to drag out the old “95% of AI implementations fail” statistic. Far more worried that they’re going to implement a model that costs a fortune and delivers mediocre results than that they’ll implement an AI driven round of end-user testing.
Turning Q&A over to the Roomba AI to find corners of the setting that snag the user would be Gud Aktuly.
And hire other people with the excess budget. Hell, depending on how badly these systems are implemented, you can end up with more staff supporting the testing system than you had doing the testing.
If they can do their work more efficiently, they don’t get laid off.
It just means a better % of edge cases can get covered, even if you made QAs operate at 100x efficiency, they’d still have edge cases not getting covered.
I was going to say, this is one job that actually makes sense to automate. I don’t know any QA testers personally, but I’ve heard plenty of accounts of them absolutely hating their jobs and getting laid off after the time crunch anyway.
They already have a really cool solution for that, which they talked about in their GDC talk.. I don’t think there’s any need to slap a glorified chatbot into this, it already seems to work well and have just the right amount of human input to be reliable, while also leaving the “testcase replay gruntwork” to a script instead of a human.
Literally not how any of this works. You don’t let AI check your work, at best you use AI and check it’s work, and at worst you have to do everything by hand anyway.
From a game dev perspective, user Q&A QA is often annoying and repetitive labor. Endlessly criss-crossing terran hitting different buttons to make sure you don’t snag a corner or click objects in a sequence that triggers a state freeze. Hooking a PS controller to Roomba logic and having a digital tool rapidly rerun routes and explore button combos over and over, looking for failed states, is significantly better for you than hoping an overworked team of dummy players can recreate the failed state by tripping into it manually.
I mean, as a branding exercise, every form of sophisticated automation is getting the “AI” label.
Past that, advanced pathing algorithms are what Q&A systems need to validate all possible actions within a space. That’s the bread-and-butter of AI. Its also generally how you’d describe simulated end-users on a test system.
I mean, as a branding exercise, every form of sophisticated automation is getting the "AI" label.
The article is specifically talking about generative AI. I think we need to find new terminology to describe the kind of automation that was colloquially referred to as AI before chatgpt et al. came into existence.
The important distinction, I think, is that these things are still purpose-built and (mostly) explainable. When you have a bunch of nails, you design a hammer. An "AI bot" QA tester the way Booty describes in the article isn't going to be an advanced algorithm that carries out specific tests. That exists already and has for years. He's asking for something that will figure out specific tests that are worth doing when given a vague or nonexistent test plan, most likely. You need a human, or an actual AGI, for something on that level, not generative AI.
And explicitly with generative AI, as pertains to Square Enix's initiative in the article, there are the typical huge risks of verifiability and hallucination. However unpleasant you may think a QA worker's job is now, I guarantee you it will be even more unpleasant when the job consists of fact-checking AI bug reports all day instead of actually doing the testing.
videogameschronicle.com
Aktywne