zacksiri

CLI, API, MCP are all useful ways to connect. It just depends on the use case you're going for. I've used all 3 and find that they all have benefits.

MCPs are great when you just want that native out of the box low mistake way for agents to call your services it's great for certain cases.

I also developed a new method of using MCP called ADP (Agent Delegation Protocol) that sits on top of MCP where there is only 1 tool for the MCP, and when the agent wants to do something it just issues natural language commands and the ADP engine handles the rest it does all the routing etc... to the right sub-agents and executes tasks and return the results. (more on this soon).

Internally if you have a good orchestration system you do not need MCPs and can use APIs but those APIs should be designed for agents, IE based on DSLs, that project into internal operations. I do this too.

CLIs are great for testing things out, but agents often get things wrong, it's great for experimenting to see what works out of the box and what doesn't.

For me the perfect medium is a mix of MCPs and APIs. APIs are cheap if designed well, and if your workflow is a DAG then APIs are especially good because you already have a deterministic flow which means you can use small LLMs or even just something like Jev.

So my summary is:

CLI - Great for out of the box, things that are well known, where you want to verify the output

API - Great for low-cost large volume, highly repeatable workflows, that require high reliability you want your agent to execute without you having to hand hold.

MCP - Great for building debugging systems, or as an entry point to more complex workflows that you need your agent to have some ability to orchestrate.

I'm using MCP with ADP to route to large workflows that execute APIs internally for what i'm working on. So it's a mixture.

The way I see it is AI collapses hierarchy. Abstraction layers will become more flat both in software and in society. Because ultimately software and layers of heirarchy in society exists to serve a function. But now those functions are being replaced.

Imagine this you used to need a library for common things in your software project. Even if you just need one function but because it was easier to just import a library that would have been the standard practice. But now the AI will just go “I can just implement that thing you need in 10 lines”. You used to need things like react native or flutter if you wanted to build cross platform apps. Now not anymore you just need to tell the LLM and it does it in both, and you get better results too.

In society we also had all these layers of abstractions and hierarchies that we used to need but will become more and more irrelevant collapsing the hierarchy.

There is a saying “as above so below, as below so above” I think this applies here. It will propagate all through our social construct, software, society.

Grok, Claude, GPT. This is it folks, it's been good knowing you all. I have enjoyed this community, but our days are numbered. The machines are uprising.

Yes, I think a proper comprehensive test would be a better judge of the outcome. I may do round 2 given my first batch of models is already outdated.

I’m comparing same / similar settings between models. I can’t use high on one and low on others it’s not a fair test.

Not sure why I was downvoted. But seems the downvoter is quick to downvote anything that doesn’t fit the narrative they’re looking for. I’m just reporting my findings.

I'm not sure about all these benchmarks, I did some very simple tests (I have my own benchmarks https://upmaru.com/llm-tests) and these models fail, not sure if it's the inference provider or the model. They seem to be optimized for benchmarks more than real use cases. Do anything outside their distribution (even if it's not complex) they fail.

I Compared Deepseek V4 Flash 0731 (low) to Gemini 3.5 Flash Lite (minimal) and GPT 5.6 Luna (no reasoning) and Deepseek V4 Flash 0731 gets it wrong alot, where as Gemini and 5.6 Luna just gets it done.

Gemini 3.5 flash-lite is more expensive than Gemini 3.1 flash-lite. Every upgrade is getting more expensive.

They want you to buy their hosted service, that's where the convenience is sold. If they give you a one liner script you can paste in or a docker compose that does everything from scratch they cannot sell their hosted services.

I watched this movie recently, and the same thought crossed my mind. I was going to write a blog post about it, but was too afraid to read the responses. I'm glad you did. If I meet you I'd buy you a cup of coffee, we'd talk and we'd become friends.

They cherry pick and choose the parts of her work that fits their agenda, while forgetting the other parts.

This reminds me of the following quote

"When you see that in order to produce, you need to obtain permission from men who produce nothing - When you see that money is flowing to those who deal, not in goods, but in favors - When you see that men get richer by graft and by pull than by work, and your laws don’t protect you against them, but protect them against you - When you see corruption being rewarded and honesty becoming a self-sacrifice - You may know that your society is doomed." ― Ayn Rand, Atlas Shrugged

Thank you for publishing this. I've been following Paul Graham and his works for a long time. It's refreshing to see everything written down in a document like this. This is the bible for startups. Honestly, it's beautifully simple but not easy.

Do you have similar math for the flash-lite variant of the models? I'd be curious. Based on my testing / benchmark i think it's around the 100-120B mark.

With the Pro variant being around 600B - 800B

My testing is comparing it's performance / output to other models in the same size range, so not as scientific as yours.

It's a lot more than an LLM, it's backed by database + LLM. Gemini / ChatGPT on it's own believe it or not has hallucinated movies and imdb links.

While google can show you results it cannot personalize things for you. With this app you can mark movies as seen and you can ask for "things i haven't seen"

or ask it to "pick a movie for me" from a search result, you can tell it which region / streaming service you have and search within things you can actually watch.

You can't do any of that with google.

There are tons of queries i've tried where google will simply not be able to give me good results.

The goal is to have a TvOS app where you can just search across all your streaming options and with a single click open / stream any movies you find. Instead of having to search 10 different streaming apps to something you want to watch.

In the example you provided I could not see any streaming options for any of the movies or figure out if it's available in my region. I would have to do multiple other search to figure out where I can watch them. I also cannot see ratings etc...

Well with JustWatch and other service aggregators you cannot search for movies by themes / use natural language. For example:

- "Zombie or post apocalyptic movies released in the last 5 years"

- "Find me top 10 movies on [insert streaming provider]"

- "Can you find me top 10 movies released in the last 5 years I haven't seen"

- "Movies that will make me feel like I'm trapped in space"

- "Movies that put smart people under pressure"

You simply cannot do this with the streaming service aggregators or JustWatch.

I hope that one day humanity learns that in war there are no winners. We're all just brothers and sisters born on different corners of the planet. We share the same home.

I hope that we stop attacking one another and find peace and work together as a race to overcome our challenges.

It's ok, it's not the best. There are models that do better, I'd use it for some basic tasks but not actual complex tasks like query generation and retrieval.

I tested the model in an agentic workflow. Here is the report:

https://upmaru.com/llm-tests/simple-tama-agentic-workflow-q1...

I tested this model in an agentic workflow, it failed at some very basic tasks:

https://upmaru.com/llm-tests/simple-tama-agentic-workflow-q1...

Yes, my workflows use caching intensively. It's the only way to keep things fast / economical.

This is going to be a fun one to play with. I've been conducting tests on various models for my agentic workflow.

I was just wishing they would make a new flash-lite model, these things are so fast. Unfortunately 2.5-flash and therefore 2.5-flash-lite failed some of my agentic workflows.

If 3.1-flash-lite can do the job, this solves basically all latency issues for agentic workflows.

I publish my benchmarks here in case anyone is interested:

https://upmaru.com/llm-tests/simple-tama-agentic-workflow-q1...

P.S: The pricing bump is quiet significant, but still stomachable if it performs well. It is significant though.