The Shire suffered terribly during the Great Plague and the Long Winter and also was invaded by Orcs at least once. The appendices say there was major loss of life. There was another very bad winter in Bilbo’s youth that was compounded by an invasion of “White Wolves.”