As far as not being able to produce machine code without even a single byte error of consequence, true, it's questionable whether that's possible. But I'm not sure that is proven impossible with LLMs? So I think it's worth testing it.

I was thinking of training it on all of the 6502 compatible machine code that's out there. There are at least 20,000 or 30,000 programs.