How is AI supposed to simulate a player, and why should it be able to determine what real people would find engaging?
I don't think it's much of a stretch to take this data over multiple games, versions, and genres, and train a model to take in a set of mechanics, stats, or even video and audio to rate the different aspects of a game prototype.
I wouldn't even be surprised if I heard this is already being done somewhere.
Whether that set is actually useful is a separate issue but someone is trying this over there for sure.
Yes, that's how games like Concord get made. Very successful approach to create art based on data about what's popular and focus groups.
I think what the previous comment meant was that there is data on how player play, and that tends to be varied but more predictable.
AI/fuzzers can't get far enough in games, yet, without a lot of help. But I think that's because we don't have models really well suited for them.
Edit: yup, it shut down nearly a year ago
Compare that to Helldivers 2 (online-only live service game, same platforms and publisher) which had a lot of personality (the heavy Starship Troopers movie vibe) and some unique gameplay elements like the strategems.
And sometimes it works; Apex Legends came out of nowhere and became one of the big live service titles. Fortnite did a battle royale mode out of nowhere and became huge.
Everything is measured and analysed and optimised for engagement and monetisation.
When you have 200 people making a game, "luck" or "art" doesn't factor in at all. You test, get data, and make decisions based on the data, not feelings.
Solo devs can still make artsy games and stumble upon success.
Where we used AI (machine learning, not LLM) was in terms trying to figure out what kind of human you would want to play with. We also used machine learning to try figure out what cohort of players you were in so we could tweak engagement.
Where LLMs could really shine, in my opinion: Gamers love to play people, not AI (now). People are unpredictable, they communicate, they play well but in ways a human could (like they don't have superhuman reflexes or speed). You can play all kinds of games against AI (StarCraft, Civilization, training of all kinds of FPS) but it isn't fun for long because you see the robotic patterns. However, an LLM might be able to mix it up like humans, talk to you, and you could probably make it have imperfect reaction time, coordination, etc. That would really help a lot of games that have lulls in human player activity, or too much toxicity.
I would be shocked if some games aren't doing this now. It seems like it still be hard to make a bot seem human, and it probably only works if you sprinkle it in.
Edit : oh yeah. A quick google search proved it : https://marvelsnapzone.com/bots/
I'm sure you could conjure up any number of ways to do that, but they won't be trivial, and maintaining those tests while you iterate will only slow you down. And what's the point? What information would pass / fail on the unit-move-and-attack test give you as a developer? It's not going to tell you if it looks good, or if it's fun.
Ultimately you just have to play the game, constantly, to make sure the interactions are fun and working as you expect.
You use a second enemy that spawns, moves towards the "enemy", and attacks.
You can easily write a 'simulation' version of your event loop and dependency inject that. Once time can be simulated, any deterministic interaction can be unit tested.
You're right that "unit test" has taken on another, rather bizarre definition in the intervening years that doesn't reflect any kind of tests anyone actually writes in the real world, save where they are trying to write "unit tests" specifically to please the bizarre definition, but anyone concerned about definitional purity enough to quibble about it will use the original definition anyway...
Like letting speed runners skip half your game. :)
The real reason? It's because writing tests is a different skill and they don't actually know how to do it.
Sounds very much like the description of a big ball of mud.
An interesting gamedev video I saw recently basically boiled down to: "Build systems, not games." It was aimed at indie devs to help with the issue of always chasing new projects and making code that's modular enough to be able to reuse it.
But taking a step back, that very much feels like it should apply to entire games, where you should have boundaries between the components and so that the scope of any such pivot is managed well enough not to tank your velocity.
Other than that, it'd be just the regular growing pains of TDD or even just needing to manage good test coverage - saying that tests will eventually need changes isn't the best argument against them in webdev, nor should it be anywhere else.
I mean, yeah, kinda.
For any given object in the game world, it's funnest for that object to be able to interact with as many other objects as possible in as many ways as possible. A game object's handles for interaction need to be globally available and can't impose many invariants—especially if you don't want level designers to have to be constantly re-architecting the engine code to punch new holes for themselves in the API. Thus, a lot of the logic in a given level tends to live inside the callback hooks of level objects, and tends to depend on the state of the rest of the level for correctness.
Modularity is a property of high cohesion and low coupling, which are themselves only possible when you can pin down your design and hide information behind abstraction boundaries. But games are a flexible and dynamic enough field that engines have to basically let designers do whatever they want, whenever they want in order for the engine to be able to build arbitrary games. So game design is naturally a highly-coupled, incohesive problem space that is poorly suited to unit testing.
Poorly suited? Perhaps, but so are certain web system architectures as well, neither is impossible to test.
I think Factorio is an example that it can be done if you care about it... it's just that most studios shipping games don't.
https://www.factorio.com/blog/post/fff-438
https://www.factorio.com/blog/post/fff-366
Of course, in their case it can actually be justified, because the game itself is very dependent on the logic working correctly, rather than your typical FPS game slop that just needs to look good.
https://www.youtube.com/watch?v=AmliviVGX8Q (kovarex - Factorio lets fix video #1)
Games have goals, and players are prone to 'optimising the fun out of games', by doing some save strategy over and over again to reach that goal, even if it's not fun. Think eg grinding in an RPG, instead of facing tough battles with strategy and wits and the risk of failure.
Even if AIs are terrible at determining what's engaging, you can probably at least use them to relatively quickly find ways that you accidentally opened that let players get in the way of their own fun.
And note, this is not AI as in asking an LLM what to do, this is more classical machine learning and deep learning.
But we did? We've come a long way from the limited XBLA catalog. It didn't happen overnight, but doubtless we wouldn't have the volume of games we have today without Unity, Godot, Gamemaker, Renpy, RPG Maker...
I'm not sure the 2 of you are disagreeing. We definitely saw an explosion of indie games. In 2010, there were less than 10 indie games released on steam per month. By 2022, there were ~500/mo, and today there's ~750/mo (I expect that the 250/mo jump around 2022 can likely be attributed to LLMs).
What's hard to say is if this increase significantly increased the number of good games. Mostly because "good" is highly subjective, but also, I think something else happens. I've been playing games for the better part of 40 years, and what I noticed, is that in that time, the number of must play games each year has largely gone unchanged, despite the industry being orders of magnitude larger than it was 40 years ago. But that is also tricky, because 2 things happen every year, our standards get higher, and our preferences get more refined.
Think of the negative reputation the Unity engine gained among gamers, even though a lot of excellent games and even performant games (DSP) have been made with it.
More competitors does also raise the bar required for novelty, so it is possible that standards are also rising in parallel.
Unity + Steam just makes this process a bit easier and more streamlined. I think the new thing is that as well as the dickwads who are trying to rip people off, there are well-intentioned newbie or indie developers releasing their unpolished attempts. These folks couldn't publish their work in the old days, because making CDs costs money, while now they can.
I don’t think we would’ve seen a Hollow Knight without Unity, built by a team of 2-3 devs.
A 90s PC can't do a complex 3d engine because it lacks the grunt. A 2020s game dev can't do a complex 3d engine themselves because they don't know how to do complex 3d.
Maybe it only got visible to the consoles generation around the time of XBLA arcade, and even that was already predated by PS Yaroze and PS2Linux efforts.
Before Unity, we had SDL, Ogre3D, SFML,... but naturally all of those require more coding skills than engines designed with UI workflows in mind.
It can do this. From Atari games to StarCraft this has been a thing since before LLMs.
> surfacing which mechanics keep simulated players engaged
This it's unclear how to operationalize. Among other things, not all games appeal to all people.
AI is insane. It can do like 10,000 actions per second lol
This is not true at all. I have never worked on games and it will take me quite a while (even months) to write a "basic" game. While I know a lot of good practices about software development and decade+ of FAANG experience, I don't know the intricacies or even the basics of game development.
I recently experienced this for a different usecase. As an experienced backend developer, I wanted to automate some javascript/browser stuff. I tried on my own for 2-3 days and had couple of prototypes but nothing actually worked. I spent 2 hours with an AI and I had a working solution. We even iterated together quickly and solved some runtime issues and the solution is working for me seamlessly now.
So, I definitely see value of AI even for coding for experienced developers like myself.
You're contradicting yourself. I promise it wouldn't take you months, unless you're just a really bad developer.
And that's the part AI is not going to be able to help you with.
Just think of the speciality in which you aren't an expert, javascript/storage/networking ...
It's not that difficult to get a base level game up and running; ESPECIALLY with modern tooling.
Each one would require a different kind of model and model technique to make, so I wouldn't be surprised that ChatGPT has issues with it. A sprite animation loop would be better done by a potentially specialized video-oriented model, for example, and the current image and video models are barely trained on that kind of video data.
It looks like retrodiffusion.ai in particular has something close.
We definitely saw an explosion of good indie games by around early half of 2010s. Whether it had anything to do with Unity is another moot point.
Maybe something else is currently holding back another bunch of good ideas in gaming. Once another threshold gets lowered, we will see another wave of good games enabled by by that, and a return to the average rate of creation again.
In the case you're working as part of team large enough to have dedicated programmers, the majority of the roles will usually be in content creation, design and QA.
"what's the point of vibecoding if at the end of the day I still have to pay a dev to look at the code anyway... I can't vibe my way through debugging, I can't ship anything that actually matters." [1]
That was the experience a lot of people had using no/low code tools, where you could make progress, but as soon as you hit a problem you are done, because overcoming it will require skills the no/low code don't teach or really support.LLMs are only different because the interface is more accessible. But all the same problems are still there. AI is not a panacea.
[1] https://www.reddit.com/r/ProgrammerHumor/comments/1mudy12/th...
Doesn't look exactly that to me. The author built a server, studied React, built a frontend, made the card game work.
Then, with most bits needed for a card game already in place, he asked Claude to alter the existing code to implement a different card game. Understandably, it took much shorter. But it would also take much shorter if a human engineer did the same.
I mean, the entire article is problematic as proof of anything. For starters, they didn't go through a design process for a game at all, they copied existing games. Then there are all these weird technical rabbit holes they went down that really weren't anywhere near "simplest path to MVP".
I just don't think there is anything to glean from this article. Like most posts about individual experiences with AI, it's functionally equivalent to, "I had a weird dream last night".
And this isn't surprising: git-style revision control hit the scene almost 20 years ago, it was like 5 years until it was totally dialed in anywhere, another 5 before elite companies had it totally figured out, and its been slowely diffusing since, today its pretty figured out. And this is harder to use right than git.
I think it would go faster actually if every product release, every OSS tool, every god-damned blog post wasn't hell bent on saying "its done, its solved, old way cooked, new world arrived".
We're figuring it out and it takes time. That's OK.
If it was done, then we'd be drowning in great software. We're not, we're breaking even, which is impressive for a big new thing 1-2 years in.
I think this is the core insight. An AI will not be able to experience a game (or anything else for that matter) remotely in the same way that a human can experience it. It might be able to guess, based on human rankings of other similar games. But AI will never be able to actually have fun playing your game.
This concept will define the workforce that comes out of this AI boom. Maybe an AI can write a document or code like a human, only based on past samples of similar behavior, but it won't be able to synthesize exactly what it means to be a human. The human element will still need to be traded on. Your value as a human cannot be replaced, you might just have to think differently about that value.
more like, more than 5 seconds.
In traditional business apps, your goal is to make your app work and look intuitive enough for a human to use. When developing a game, you have a few extra goals, it also has to be fun, rewarding and different enough from other games that came before. It feels like the former group will be much easier to judge by non-humans than the latter.
Much the same as we do today in games and film both, something saccharine and mediocre built to appeal to a wide majority. Worse, if this process becomes streamlined and widely accessible, you're competing with a hundred other saccharine and mediocre built to a wide majority games. AI generated shovelware, it's like the shovelware of today where anything remotely popular generates dozens of cheap clones, but with AI.
The best games take risks and aren't min/maxed.
Just try to implement, for example, a hexagon-based isometric game. There are no off-the-shelf implementations -- you'll need to redo the pan / zoom / click controls yourself, you'll need to implement the pathfinding, map layers, interface state machine etc etc etc
This is still not an easy task -- to build a somehow complicated game. If you're building a platformer -- sure, that's doable. Strategy/4X/RPG? That's different.
Sure there's off the shelf implementations.
Off the top of my head I would suggest starting with evaluating godot 4.
They have isometric view, pathfinding, and all of the rest you are mentioning.
I can't say whether it's me who's stupid, or it's just not very easy to make good UI in game engines. I don't say that's not doable of course -- i'm just saying one would need to invest quite a bit of time to work out how to do this.
That’s why we’re not suddenly drowning in brilliant Steam releases post-LLMs. The tech has lowered one wall, but the taller walls remain. It’s like the rise of Unity in the 2010s: the engine democratized making games, but we didn’t see a proportional explosion of good game, just more attempts. LLMs are doing the same thing for code, and image models are starting to do it for art, but neither can tell you if your game is actually fun.
The interesting question to me is: what happens when AI can not only implement but also playtest -- running thousands of iterations of your loop, surfacing which mechanics keep simulated players engaged? That’s when we start moving beyond "AI as productivity hack" into "AI as collaborator in design." We’re not there yet, but this article feels like an early data point along that trajectory.