They’re probably using it to express incredulity.

“The mechanic told me it would cost $5000(!) to fix, so I decided to scrap the car instead.”