This is a much better benchmark as a concept (without knowing the details of how the problem is framed to the agent) than some of these ridiculous things the frontier labs are reporting (vending machine simulator? come on).

Can you report more in depth at stumbling blocks? It's a surprisingly linear curve.

I'm going to try it on a local run.

I wonder if there is room for "recommendorio" where it gives you AI advice on your current factory.

Is the model pre-trained with any knowledge of the game?