This is just my opinion, but I feel like stuff is way more likely to be actually be accomplished if there's a tangible goal. A goal, even an arbitrary one, helps you actually figure out the limitations of a theory or design, and if you don't have a goal it is far too tempting to handwave away important questions and live in theoretical land.
It does sadden me that these days their designs are much more restricted, as I enjoyed the wacky ideas we would occasionally see when the rules were less restrictive.
Even in weird ways: my sister can afford to pay her rent/food herself while studying because she prepare meal plans for a water polo team, which helps her study and find ideas for her college projects. Hopefully she can get through her studies without working in a kitchen again.
The Apollo missions flew on Rockets designed and Manufactured by Boeing (S-IC), North American (S-II), and Douglas (S-IVB).
The lunar lander was designed and manufactured by Grumman.
That said, a lot of engineering goes into setting good purchasing requirements and assessing the results.
No one has been arguing that NASA scientists are better than private sector scientists. The argument is that having a seemingly-stupid goal that takes a lot of work can spawn off legitimately useful technologies that would never have been targeted by the private sector because moonshots aren't good investments. The theory is that it's a good way to very quickly iterate through many different ideas.