Personally I don't think this would be hard to simulate, for instance fear would be marked with increased cursing, decreased accuracy, increased error, and a potential for full scale running away. Complacency would mean that guards don't check every corner on their patrol routes, have smaller visual cones, and are slower to react when shit hits the fan. Confidence could be marked by seeking cover less, drastically increased accuracy, and so on.
So yes, whilst in the distant future we could have truly "intelligent" AI which would lead to unpredictable gameplay, we at present have the tools to create enemies that can provide a much richer and more vast array of feedback to the player's behaviour.
Working in game design with complex AI, I can tell you that the last thing players ever want is rare behaviors. They tend to think of these things as bugs even if they are simply that, rare behaviors.
Then again, I've been working in game systems where the rare behaviors are sometimes positive and sometimes negative for the player. Given this, the negative ones stand out as "bugs" even if they are not.
Enemies set of motivations: kill the player
Knowledge base: 1) An organism can be killed with an excessive amount of directed energy (heat, force, etc). 2) The player is an organism 3) Some general rules about the physical world (how gravity works, how fire works, what switches and levers activate which objects, blah blah blah).
Now, the enemy might take some pre-programmed path to kill the player (like fire a gun at the player). Or, if circumstances dictate that it is best, they might produce a more "creative," emergent approach. Say, for example, the enemy pulls a lever. This was never added in by the programmer. The lever activates a trap door, above the players head, and a bunch of boulders crush the player. Switch action -> directed energy -> kill organism -> player death.
In shorter words, it really comes down to the programmer determining an effective knowledge base (abstraction), and letting the AI run wild with reasoning. :) This is not to say machine learning wouldn't be the icing on the cake, but you could just as well let enemies (or NPCs) have the knowledge before the game is launched without having to learn it.
Do people really think that it's a matter of CPU time goes in and intelligence comes out?
People seem to think that if we were not spending so much time on graphics, that we would have amazing AI by now, but it's simply not the case.
The reason we use simple state based AI's for the most part comes down to controllability.
In order to actually design a fun gameplay experience you need to have ways to tweak the AI to behave in the way that the player will find fun. You want to be able to ask "Why did that character not throw a grenade in this situation?", be able to find the answer, and tweak it.
The techniques that come out of AI research don't expose parameters in this form, and so are essentially useless for real game AI.
Probably the biggest CPU use from AI today are perception checks, like can entity X see entity Y from it's location. Checks like these tend to cross cut into engine code and just feed into the very simple state based AI.