A lot of people believe that Anthropic lose money selling tokens to customers because they are subsidizing it for growth.

But that has zero effect on revenue, it only affects profit.

I wrote awhile ago on here that he should stick to his domain.

I was downvoted big time. Ah, I love it when people provide an example so it can finally be exposed without me having to say anything.

Unfortunately this is a huge problem on here - many people step outside of their domains, even if on the surface it seems simple, but post gibberish and completely mangled stuff. How does this benefit people who get exposed to crap?

If you don't know you are wrong but have an itch to polish your ego a bit then what's stopping you (them), right.

People form very strong opinions on topic they barely understand. I'd say since they know little the opinions come mostly from emotions, which is hardly a good path for objective and deeper knowledge.

Yeah it's the caching that's doing the work for them though honestly. So many cached queries saving the GPUs from hard hits.

How is caching implemented in this scenario? I find it unlikely that two developers are going to ask the same exact question, so at a minimum some work has to be done to figure out “someone’s asked this before, fetch the response out of the cache.” But then the problem is that most questions are peppered with specific context that has to be represented in the response, so there’s really no way to cache that.

From my understanding (which is poor at best), the cache is about the separate parts of the input context. Once the LLM read a file the content of that file is cached (i.e. some representation that the LLM creates for that specific file, but I really have no idea how that works). So the next time you bring either directly or indirectly that file into the context the LLM doesn't have to do a full pass, but pull its understanding/representation from the cache and uses that to answer your question/perform the task.