That would be sensible if the traffic signs were in English.

Can you read Chinese? Can you identify what this traffic sign means? https://upload.wikimedia.org/wikipedia/commons/thumb/6/68/CN...

How about Japan?

What does a sign that says private road, residents and guests only.

Traffic signs have symbols and shapes. You are allowed to drive in the US with an international drivers license if you don’t speak English. Are they going to arrest someone who doesn’t speak English and got a license in another state?

Only if being illiterate also forbade you from driving, which it does not. You don't need to read the law to follow the law.

Well, there's a written exam.

Which can be completed by someone reading it verbally and writing down their answers, pretty much the same thing as financial and legal documents can.

Traffic signs are readable by almost anybody regardless of English language skills. A vision test is much more safety-valid than an English language test.

I disagree that traffic signs are readable regardless of language skills. Yes, it's just a matter of developing recognition for simple pictorial signs. You just have to learn it. If I put a French "No Vehicles" sign in Florida, nobody is going to have a clue what it means, even though there are no words on it, and that's dangerous.

Not recognizing or incorrectly interpreting "Crash I-9 N/B Exp Right 2 Lanes Closed Merge Left 2000 ft" is also dangerous, right?

Well countries have been willing to do reciprocal agreements with the US and other countries since 1926.

https://internationaldrivingpermit.org/what-is-an-idp/

That level of English would be considered below A1. Just being in the US for a few months would give you that level of English even without any other education. So you're conflating "can you read about 2 dozen English words" with passing an English exam - let's say B1.