No doubt. I wouldn't take away anyone's bike (unless I was their child or spouse, maybe), but I'd worry about old "frail" people on bikes. Their strength and flexibility declines (in general with individual exceptions), and a fall can debilitate or kill old people. Glad to see that three-wheelers are an option.
I remember being impressed by the fact that while this guy had so much trouble bending down, he was riding a (regular) bike.
I've also seen 'seriously' pregnant women bike around, men/women with a kid strapped to the front and back of the bike, with sometimes a third kid in a kart behind the bike. I also regularly see groups of elderly people on sports bikes (often with some electrical help though), as well as really young children.
There are plenty of people who really can't ride bikes, but probably far fewer than people from non-biking places assume.