gmarkwa

I see almost no discussion about the use cases for extremely fast diffusion models like Mercury which can apparently reach 1,000+ tps throughput, at an intelligence level around 32 on the Artificial Analysis Intelligence Index (according to Inception's webpage). Obviously this is a dumb model, but that level intelligence is close to Luna at Low effort level, which is good enough for basic tasks. I think these fast models open up a world of possibilities, especially as they get faster. I imagine workflows where, instead of taking time to write a prompt, I can rapidly iterate on the output by clicking through options. I'm wondering if anybody else is thinking about these possibilities?

Yeah maybe I'm conflating an "agent with tools" with a chat feature. Just because I'm using an agent with tools doesn't mean I need to chat with it, I could just click a button and have it do things, in the same way that I click a button and a driver with tools goes and gets my food.

Could you give me an example? Like a real example of an LLM based application with no chat? Now that I think of it something like Tavily comes to mind as an example.

I mean they can be deterministic, thinking machines lab made deterministic LLM's: https://thinkingmachines.ai/blog/defeating-nondeterminism-in....

Although obviously I'd rather they stay non-deterministic