My CS undergrad school used to let students look up documentation during coding exams. Most courses had a 3-5 hour coding challenge where you had to make substantial changes to a course project you had developed. I think this could also be the right response to LLMs. Let students use whatever they want to use, and test true skills and understanding.

FWIW, exams testing rote learning without the ability to look up things would have been much easier. It was really stressful to sit down and make major changes to your project to satisfy new unit tests, which often targeted edge cases and big O complexity to crash your code.