Turns out the summit of Mt Fuji has civilizational amenities, and there's actually a vehicle that goes up to the summit via a trail.
Different people react to altitude differently.
When we went to Peru, we flew in to Cusco which is around 11,000ft which seems like "not that much higher"; however, the decrease in pressure and O2 content is exponential, the the transition between "no problems" and "some problems" may be abrupt. My wife is fitter than I am, but seemed to have had more issues on our first few days, despite us both taking Acetazolamide (although we didn't bother until after arrival).
Another factor is that speed of ascent factors into onset of sickness; flying in to a high altitude can be worse than ascending over a period of days, as it takes time to acclimate.
Physical fitness doesn't seem to have a lot to do with how a given person will respond to altitude. I believe women and older people are slightly more likely to respond poorly, but it's kind of a crapshoot.
We hiked over 15,000ft but it definitely sucked at times -- tingly hands and feet related to the drugs, I believe.
Funny thing was, after > 1 week at that altitude, we both stopped taking our drugs the morning of our departure. Our flight was delayed by a few hours and by 1pm I started feeling the onset of symptoms again, wishing I had taken my dose that morning.
Side note (I'm not saying this applies to above comment): I'd caution anyone to take 12,000 feet seriously, it's a serious altitude and you want to think in terms of turnaround times and eventualities, not "I'm getting to the top no matter what".
For example, I climbed Mt. Massive in Colorado (second highest peak and >14k feet) without any special training or problems (I normally live near Washington DC). If you are in reasonably good shape (i.e. you run regularly) then you should be fine. Yeah, I stopped to catch my breath a little more frequently the higher I got, but it was a totally manageable 8 mile round trip.
The achievement is only undermined by the fact that there is a literal queue of people in front of you for the entire ascent, and by the fact that after completing the climb you are greeted with a vending machine at the summit!