In my own experience with openAI's gpt-4o, when feeding with long input, the model simply 'strips' the starting content and answers, while it should output error for more 'accurate' feedback. I switched to my own deployed model for that purpose.

That's a product choice from openAI, not a problem with LLM's long context window input capability.