But over long enough time it _must_ be somewhere where somebody can provide Computational power and evergrowing storage capacity, right?

So.. not anywhere.

Or can the data be available in a highly fragmented fashion _and_ still be useful? I just can't imagine any content system that does not end up being centralized or highly inconvenient.

If someone cares about a piece of content's availability on IPFS, they can host it and anyone in the network can pull it from them without having to know a priori that they provide the content. That's what I mean by "anywhere". Anyone can provide any content they want, from anywhere, on whatever infrastructure they want, as long as there's a communication channel open with the rest of the network.

If you care about high availability of a piece of content, you will dedicate infrastructure to hosting it on IPFS, just like you would on the Internet.

Is there any need for the Blockchain for that? Like... Setup a server and host the content. I can't imagine it being simpler than that.

Why would I engage in the dance of crypto just to host some content no one will read anyway?

What's IPFS?

IPFS doesn't use a blockchain nor crypto. Conceptually the Internet is a set of protocols that together map (data-identifier, provider) -> bytes (e.g. GET https://<provider>/<data-identifier>). Similarly IPFS is a set of protocols that map (data-identifier) -> bytes.