Mostly that, but as a member of a family that hunts growing up in an agricultural region (SW Missouri) I also know that the process of converting to agriculture radically changed the environment, especially the fauna. By and large a lot of wild flora continued to exist on unfarmed land, but that provided a lot less cover etc. for animal species who's various requirements got zapped. Nowadays hunters work with farmers to e.g. leave enough quail habitat at the edges of fields.

Another thing that happened was that apex predators of domestic animals were suppressed and/or replaced by other predator domestic animals like the dog and cat. In general for lots of reasons the populations of humans grew substantially beyond what wild game could support.

Although deer in the US are something of a special case. They're really thriving, we could harvest quite a few more (and in a number of places desperately need to), heck, they're the most deadly animal in the US (through car crashes). This of course wouldn't happen in countries where meat was at more of a premium and/or where the population density was a lot higher.