Pretty close. It's a smooth map with smooth inverse. If you've chosen chosen coordinates, it will transform them for you.

I think most physicists probably have a reasonable intuitive idea what a diffeomorphism is, since most have probably learned something about relativity.