News for hackers. Stuff that matters.

A comment on I benchmarked GitHub CLI, MCP, Tool Search, Code Mode so we know the differences.

Re:I benchmarked GitHub CLI, MCP, Tool Search, Code Mode so ... by hightechmind on Monday March 23, 2026 @06:40PM (#47496127)
The concurrency finding is the most striking part — MCP forcing sequential API calls because of schema retransmission between calls is a structural penalty that compounds exactly on the multi-step tasks where you'd most want MCP's composition benefits. The ci_failure_investigation example makes it concrete: 15 turns vs 3, 12× cost difference.

Have you looked at whether client-side schema caching could recover that? If the agent doesn't re-transmit tool definitions on every turn, the sequential vs parallel gap should narrow significantly.

[ Parent | on HN ]
Re:I benchmarked GitHub CLI, MCP, Tool Search, Code Mode so ... by akane8 on Monday March 23, 2026 @06:50PM (#47496200)
Technically MCP tools can be called in parallel as well, but it seems the agent is generally less likely to do so.

In the evaluation I did account for prompt caching, so the multi-turn penalty is already minimized, yet it was still significant enough to make the difference.

[ Parent | on HN ]