These are blocks designed by us. I put now basic configuration of the block as separate Vivado project to git. That’s the pain...
Not sure how much you have looked into it, but the tcl scripts can be quite handy for some things. You can also drive quite a few things from the command line. So depending on what your doing you could just have some text files instead of a whole project being pushed to a git repository.
Or are you talking about managing blocks that you or team have designed? That's annoying especially when you have all these blocks with certain performance characteristics such as power use, propagation delay, area ect... So you spend time simulating a design with each of these blocks till you get the one that meets the specs for the larger design, but saves the most space so you can put more things else where. It's nice though that some of this stuff is getting automated, but it's still a pain with analog designs.