All the things you mention would indeed take priority, and we probably spent days sleep deprived at times, making up for it with extended long sleep when there was opportunity.
I think our lives were not nearly as difficult or demanding as some imagine, however, especially when you factor in the social and tribal factor. Sleeping in shifts would allow for plenty, attacks were not constant, especially with good shelter, and shelter only takes so long to construct, if that's necessary at all.
Remember, we started in an environment which provided everything we needed already, and one we were evolved for.
If you look at the lives of other apex predators like lions, bears, and eagles, their lives are pretty good in this respect, and their biggest threats are either their own species or humans.
Exercise and absence of artificial light would probably have us sleeping much more deeply and much more quickly
In modern hunter-gatherer societies there is also a lot of free time and play, so perhaps there would also be less stress to recover from at night
I think you were right to consciously remove those things from its access.
Sleep was always an essential part of the equation, however.
Try to imagine how you lived 10K years ago.