If it were possible to create a superhuman fair playing AI then the Civ bonuses themselves would create personality (in fact often strongly related to the desired personality characteristics of each civ!) even without additional 'hard coded' imperatives, which would of course also be possible.
And in this scenario create a compelling weaker AI would be much easier than trying to create a compelling strong AI (from a very weak base). There will need to be some 'move' evaluator function and simply picking weaker 'moves' would so fine. If that creates uncompelling AI then we could get back to cheating and impose resource penalties, but with much more compelling play. E.g. - kind of like playing a strong chess engine with knight odds.
Look up different game AI architectures: scripted, rule-based, finite state machines, behavior/decision tree, utility, planning, hierarchical task networks, and machine learning. Hard-coded imperatives are very difficult on highly autonomous architectures, as is complex systemic behavior in authored ones. Consider how hard it is to make an autonomous AI system like ChatGPT implement one simple hard imperative: not revealing the system prompt.
Weak AI and strong AI are ambiguous terms. Most junior-level software engineers can create very complex gameplay systems if you're talking about complexity. It’s much harder to build simple ones that effectively create the illusion of intelligence on limited hardware. If you are talking about difficulty, then challenging AI is no harder to make than easy AI. Usually, game designers will naturally create hard AI as they play the game so much they become experts, and player testing is used to make it easy again.
As you describe it, the move evaluator is a utility function approach that is neither very authorable/personable nor very autonomous. It is pretty limiting on both fronts, so it’s rarely used. A simple rule-based approach is more straightforward to balance and personalize for each agent/archetype. And difficulty scaling or balancing the game differently doesn’t require what you propose. They just chose not to do the balancing you want.
There’s nothing wrong if you prefer a utility approach. We all have our preferences, and I like some pretty clever AI systems like GOAP that aren’t always a good fit for games like Civ. They are cool and smart systems but players rarely respond better to them than simple FSMs or rule-based game AI, which are easier to author. The utility approach is somewhere in the middle and could also be made context-specific with enough separate utility functions the agent would evaluate. But it is so much more work than to define simple if-then rules. :)
Take one of the most controversial features of CIV 5: one unit per tile. On the face of it, this is a great feature that opens up interesting tactical opportunities. Blockading, the importance of terrain, formations etc.
Unfortunately, whether due to unwillingness or lack of competence, the AI was completely befuddled by this. It'd make big armies, and then be fundamentally incapable of moving them. It would shuffle units one by one around endlessly and get stuck in the smallest of congestions.
This is not about whether an AI should play to win like a human or it should roleplay. I agree - and the majority of playerbase agrees - that the CIV AI should primarily roleplay. We all remember the frustrating double-crossing bastards during the early days of CIV 5.
This is not about the AI being complex or not. This is about making a game where the AI is competent, and this is not just the AI being some super smart intelligence in absolute terms. It is (mostly) about designing the game such that the AI can appear competent and pose a threat where the human player can feel like they are playing against an opponent that can test their skill, instead of just trying to beat some pre-determined math that is stacked against them.
CIV 4 was a game built in an era where computers were orders of magnitude less capable, and yet the AI in the game poses a much better and more competent appearing threat than in the current iterations. Not because it was more advanced, but because the game was built such that the AI didn't need to be that advanced. The AI still cheated of course, but it only needed a few simple heuristics and rules to build doomstacks that appeared "smart". A simple A* was enough for it to navigate this doomstack to your capital and make you sweat.
Now we have very complex tactical battles with positioning and formations, but this would require vastly more complex AI to do well. Unfortunately, the AI has not kept up, so we are left with opponents that appear dumb as bricks because they don't play the game the human plays. The AI simply spawns resources and armies into existence out of thin air in ever more blatant ways. The less the AI plays by the same rules the player has to, the less it feels like a story and more like a math puzzle.
I fully believe that the great mass of the bell curve is happy with the bad AI, but for those of us who are a little better at the game, we will keep complaining about how terrible the AI is.
It can also lead to the appearance of buggy behavior if you don't understand the implementation. For instance I think Firaxis did a much better job of the AI in XCom (in that it's enjoyable to play against at least), but there too you kind of need to understand the implementation or you might wonder about why, for instance, enemies seem to be magically teleporting around the map. The answer, of course, is that they indeed are.
Of course creating a non-cheating AI that's competitive against skilled humans is extremely non-trivial. But if you at least work on improving the AI you can tune down the amount of cheating necessary to create a challenge, which means the experience can feel much closer to 'organic.'
It matters for the same reason that people don't want to play against a human-like AI, because that AI would be a double crossing bastard who tries to win at all costs. They want to play against a roleplaying AI who tries to appear like Ghenghis Khan or Tutanchamon. But if I see my sworn enemy Attila the Hun spawn in an endless stream of battering rams out of thin air, and yet is so utterly incompetent and dumb that it cannot navigate a simple pass between mountains, that's a worse transgression against my suspension of disbelief than if he was just plain bad at the game.
Firaxis might simply be balancing with formulas like y = mx + b, where the AI starts with a bonus b and gets better at a rate m through the age/game. I am only speculating, but perhaps they want the AI agents to seem harder at the start and make it more likely that the player overcomes them over the game. The feeling of dominating an opponent is one component of a design pillar in many games called "power fantasy," which players love. Dominating them from a position of the underdog is even more rewarding. Whether this plays as well as never giving the AI agents an "unfair" advantage is another question, but this advantage is a part of the Civ formula now. And there is a bit of inertia around design traditions in game series - many things are built around them that wouldn't work and would have to be redone if they changed. Perhaps they didn't want to spend time here.
One way or the other, this is a matter of balancing and not big-picture game AI design, architecture, or systems as much. So, if you are only concerned with cheating, then set all of what I said about these big-picture things aside. However, the comment I was responding to was more about the overall quality of AI systems.
I hope the context I've given anyone who reads this lets them communicate with Firaxis about the cheating problem in a more direct and constructive way. Saying that AI is bad overall from the perspective of system modernity does not express that sentiment and would confuse them. I am happy with the game AI in Civ in the same way most programmers are okay with bugs in software. It's not perfect, but I understand why it is this way. You can challenge the cheating in their forums if you'd like.
The AI certainly does tell you what it's thinking: "Duuuuhhhhhhhhhhh"
In most strategy games, rule-based AI is easier to author, contextualize, and theme. Players usually prefer AI agents with memorable and varied traits instead of excellent high autonomy (complex utility, HTN, ML) AI. They like evident strategy more than complex strategy. Smarter and harder AI sounds appealing on paper, but it’s often less fun for most players who want a predictable, beatable opponent.
High-fidelity human-level opponent AI is only really appreciated in eSports, where it serves a training purpose. However, eSports people have fun in an aim trainer—they are not the “normal” gamer.
Most people want “really weak chess” and wouldn't enjoy hard or complex chess. They like game AI opponents acting in simple and vulnerable patterns they can outsmart and feel clever. And the more they dominate the agents by learning them, the more they like the game. Even if they say they don't, the user testing shows they do. The cheat bonuses that are super easy to win against are probably intentional :)
There is also another reason for building simple game AI — the illusion of intelligence is not created by complexity but rather by anthropomorphization/personification. It's the same as with LLMs — the big breakthrough was when they started speaking like humans, and all further complications aren't adding much to the broad and intuitive feeling of their intelligence. Similarly, a game AI agent that tells you what they are thinking achieves more perceived intelligence than an AI agent that is actually more intelligent. With this in mind, game developers don't waste much time on deep intelligence. This time is better used making agents bark, animate, display human reactions, show context awareness, etc. It is very much about the agents being showhorses, not workhorses.
I say this from pretty long AAA game AI design and architecting experience. It’s wiser to do what Firaxis did.