The instrument itself plays differently from anything else in an orchestra. It really is a bizarre instrument. And it's an absolute riot to play. The double reed, woody sound, and enormous four-octave range make playing the bassoon a really unique endeavor. The instrument is built out of tune with itself. It takes a certain kind of... distinctive individual to really connect with the instrument. Ask any concert musician - they'll nod and grin when you mention those odd bassoonists.
The closest instrument I can think of in terms of rarity and unusualness is the French Horn. It's really the odd duck of the brasswinds. French hornists are a little more common than bassoonists, but then most instrumentations call for a few more horn players.
Man, this has been a trip down memory lane. What might have been....?
I'd guess the oboe as the next least popular, but that's probably just because of the joke.
What's the difference between an oboe and a bassoon? The bassoon burns faster.