I think there's space for a Heroku/Netlify/Firebase style service. Somewhere between large scale and none.
It took me a few weeks and some talented/passionate engineers to figure out embeddings and vector DBs and searches. And another few weeks to set it up to a production level. Markdowns are cute, but every second of processing time is the difference between people using it and people dropping off.
If you have something that's just plug and play, and fairly cheap to start with, that would help a lot of people. Some would just stay locked in rather than hire an engineer to deal with it.
I agree that it is a hard problem at scale. But at that scale, companies should be able to build their own memory layer for their agent's use case, instead of relying on a memory layer which is built for a more general use case.