Makes sense.

But caching, for instance, doesn't need to be it's own lib does it? I don't want 'semantic caching' I want to cache the exact same query and I can do that without being LLM specific:

    from joblib import Memory

    @memory.cache
    def call_chat_completion_api_cached(max_tokens, messages,temperature):
    ...

I mean, I guess then I might want to store that somewhere central like redis and maybe slowly I need a specific cache tool. So I get your point. It's helpful to see the possibilities of approaching these problems.

But it also does feel like an land grab of supporting libs and infrastructure.