ursaguild

This is really cool to see. Being able to play the token generation was awesome. Amazing job with breaking down how to think about these models. This made the idea of Jev/decision models really easy to grasp for me. The idea of calibrating the model was helpful. I thought this was a great overview.

Especially with a client, consider the word choices around "learning". When using llms, agents, or rag, the system isn't learning (yet) but making a decision based on the context you provide. Most models are a fixed snapshot. If you provide up to date information, it will be able to give you an output based on that.

"Learning" happens when initially training the llm or arguably when fine-tuning. Neither of which are needed for your use case as presented.

Ingesting documents and using natural language to search your org docs with an internal assistant sounds more like a good use case for RAG[1]. Agents are best when you need to autonomously plan and execute a series of actions[2]. You can combine the two but knowing when depends on the use case.

I really like the OpenAI approach and how they outlined the thought process of when and how to use agents.

[1] https://www.willowtreeapps.com/craft/retrieval-augmented-gen...

[2] https://www.willowtreeapps.com/craft/building-ai-agents-with...

I was under the impression ACL supersedes KQML and was proposed by FIPA?

This article was written a few weeks after MCP was released and touches on why MCP is important. While I guess you could argue that technically there's nothing to it, protocols such as MCP is addressing a missing need to standardize interactions between your ai app and another service. Code needs to now be written for users, devs (apis), and ai.

https://www.willowtreeapps.com/craft/is-anthropic-model-cont...

smolagents by huggingface would be more of an agent framework. If it was discussed we would see smolagents/llamaindex/pydantic/etc with frameworks on figure 2. Several frameworks were left off in this paper as it focuses more on the protocols.

https://huggingface.co/docs/smolagents/

The real benefit I see from mcp is that we are now writing programs for users and ai assistants/agents.

By writing mcp servers for our services/apps we are allowing a standardized way for ai assistants to integrate with tools and services across apps.

Just saw that this was built for a hackathon. Huge kudos and congratulations!

I like the idea of more comparisons of models. Are there plans to add independent analyses of these models or is it only an aggregation of input limits?

How do you see this differing from or adding to other analyses such as:

https://artificialanalysis.ai

https://huggingface.co/spaces/TTS-AGI/TTS-Arena

https://huggingface.co/spaces/hf-audio/open_asr_leaderboard

https://huggingface.co/spaces/TIGER-Lab/GenAI-Arena

Great work on all the aggregation. The website is nice to navigate.