There are coil antennas, but they work magnetically. The 40m straight wire works electrically. The key difference is that the electric antenna interacts with more of the wave, so it's stronger in the far-field, while the magnetic antenna interacts more localized. The latter is why/how NFC works at 13 MHz (23 m wavelength) with an antenna that fits on a credit card.

Old school 27MHz CB radios would have 1/4 wave antennas 2.75m tall. In practice, for vehicle mounted use, they'd almost always have bottom or centre coils to allow overall lengths down in the 1.5-2m sort of range, with compromises accepted there for practicalities and convenience. Fixed base stations would often use a "proper" 2.75m antenna though.

Not really, atleast not the classic designs and if you want to transmit. You can bent them a bit, but the total lenght has to be around the quarter wavelength