I agree, GPT3.5 has been working insanely well for me when making overpass turbo queries. I guess there's lots of training data in the dataset for it. Made a quick 10 liner that takes in a bbox & search prompt -> GPT3.5 -> overpass turbo -> geojson

i.e. "banks in my area" a la google maps here: https://i.imgur.com/pSRn1xo.png

In my experience it seems like prompting for older technologies works much better on large LLMs, i guess it makes sense considering that there is probably more crawlable documentation out there

EDIT: someone in this thread linked a tool which seems to do this: https://github.com/rowheat02/osm-gpt