So either pre-reproductive mortality is through the roof, but never mentioned, they haven't been there long[1], or something bad happened recently (before the events in the LoTR)[2].
[1] Except Hobbits settled the Shire about 3000 (edit: 1500, I can't count) years before the books.
[2] The idyllic setting, lack of traumatic narrative and their 1500-year settlement somewhat implies they haven't been through some kind of brutal population bottleneck recently.
Well if we're in this framework of assuming middle earth functions (more-or-less) like our own pre-modern world, then pre-reproductive mortality is enormous. If the Gamgees had their children in Europe in the Middle Ages, then only 6-7 of their children would survive to the age of 10.
Considering that a Hobbit male is only considered an adult at 33 years old, I'd assume that hobbits in general reach sexual maturity later than humans as well, giving any particular hobbit a higher chance of dying before their first child than any particular human.
Plus we don't know if Rosie is an outlier or not - plenty of real-life women have had 10+ children, especially in pre-industrial societies. The average woman in pre-industrial Europe had 6-7 children but effective fertility was barely more than 2 due to pre-reproductive mortality - we didn't see a population explosion in Europe until the 18th century, rather the population very slowly ticked up over centuries and crashed back down in various calamities.
100,000 is still pretty low, but as far as I know there are events that would have caused population shocks, including major crop failures due to bad winters in Frodo's memory, and a few generations previously there were both plagues and even a mass migration of the entire Hobbit population to Hobbitton due to attacks by Orc bands.
In medieval times, people had lots of children, but most of them died very young. Of course this doesn't fit into Tolkien's idyllic description of the Shire, so it's not mentioned, but I guess families with 13 children who all survived to adulthood were the lucky exception...
But that might be just a UK naming thing since "city" means something special (and isn't actually especially related to size: the smallest one is 1600 people).
But yes, perhaps it's not really a "small" town but just a town.
a) Historical populations were much smaller than current population, so 'big' scales similarly. The world population around 1700 was about 10% of current population.
b) And maybe the bigger effect, we've had MASSIVE urbanization in the last 100-150 years. In 1500, Europe had urbanization rates well below 10% [https://www.britannica.com/topic/history-of-Europe/Demograph...], compared with 72% in 2015.
So not only were there 90% fewer people, but most of them didn't live in cities. Paris, the largest city in Europe, had a population of around 150k-250k in 1500, depending where you look. [https://en.wikipedia.org/wiki/Paris_in_the_Middle_Ages#Popul...]
Maybe Hobbits really just are that resilient and cheerful that they just carry on despite massive depopulation events that would make the Black Death look fairly tame.