I think it's also a matter of "shape". Like, GPT4 solves one "shape" of problem, given tokens, predict the next token. That's all it does, that's the only problem it has to solve.

A Civilization AI would have many problem "shapes". What do I research? Where do I build my city, what buildings do I build, how do I move my units, what units do I build, what improvements do I build, when do I declare war, what trade deals do I accept, etc, etc. Each of those is fundamentally different, and you can maybe come up with a scheme to make them all into the same "shape", but then that ends up being harder to train. I would be interested to see a good solution to this problem.

You can constrain LLMs (like LLAMA) to only output certain tokens that match some schema (e.g. valid code syntax).

I don't see why you can't get a LLM to output something like "research tech332; build city3 building24".

I am 100% certain that the training of such an AI will result in winning a game without ever building a single city* and 1,000 other exploits before being nerfbatted enough to play a 'real' game.

(That doesn't mean I don't want to see the ridiculousness it comes up with!)

* https://www.youtube.com/watch?v=6CZEEvZqJC0

If you train the model purely based on win rate, sure. Fortunately, we can efficiently use RLHF to train a model to play in a human-like way and give entertaining matches.

I knew it, I knew it! It would be a Spiffing Brit video.

That guy is a genius at finding exploits in computer games. I don't know how he does it, I think you need to play a fair bit of each game before you find these little corners of the ruleset.

Idk maybe he uses some sort of fuzzer

But wouldn't this be amazing for the developer to fix a lot of edge cases/bugs?

Maybe, maybe not. The stochastic, black-box nature of the current wave of ML systems gives me a gut feeling that using them like this is more of a Monkey's Paw wish granter than useful tool without a lot of refinement first. Time will tell!

> I’ve been dying for someone to make a Civilization AI.

Would love to see someone to make an AI that can predict our economy, perhaps by modeling all the actors that participate in the economy using AI agents.