"All of this hinges entirely on your direct manager (and to an extent their manager) being an actually good manager" I've found this to be a pretty tall ask at most companies. IMO If companies got rid of most of the performance and goal setting, most lower level managers could just convert to project management and teams would be a lot more effective.
If you're not doing that work, either someone else is going to do it or it'll cause issues down the road. Look at it this way: you can either be grateful that so many folks are wanting to help you in your effort and coordinate that effort, or you can stick to the code and complain that someone else is trying to steal your credit.
All of this hinges entirely on your direct manager (and to an extent their manager) being an actually good manager, and not a microcontroller, pass-the-buck-er, or an empty chair.