Yup. Until there's selection pressure working against you once again. Imagine if society broke down completely, so you had to fight for the limited amount of food that was available?
Somewhat related: it was interesting when my kids were born; they both had to be C-sectioned. I asked the obstetrician about it, and yes, bigger heads are becoming more and more common because mother+child no longer just die when they do occur. We really are removing selection pressures!
A great example of this is me; I'm pretty amazing; I can move around my environment, make a living to a sufficient degree to cover costs of food and shelter, and somehow even have a fiancée and manage to keep another organism alive & fed. However I'm not an expert at everything, I'm pretty bad at most sporting based things (even compared to other, existing humans; not some theoretical concept of perfection), and I spend about a third of my life doing nothing (sleeping), despite this not necessarily being required (e.g. Dolphins can sleep half their brain at a time, allowing them to avoid this inefficient downtime).