Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep

Re:Show HN: Semble – Code search for agents that uses 98% fe... by esafranchik on Sunday May 17, 2026 @01:49PM (#48171250)

Two follow-ups:

1) How do you compare accuracy? by checking if the answer is in any of the returned grep/bm25/semble snippets?

2) How do you measure token use without the agent, prompt, and tools?

[ Parent | on HN ]

Re:Show HN: Semble – Code search for agents that uses 98% fe... by stephantul on Sunday May 17, 2026 @01:51PM (#48171278)

1) yes! It’s not accuracy, but ndcg 2) we assume that if the agent gets the correct answer in the returned snippets it does not need to read further

[ Parent | on HN ]

Re:Show HN: Semble – Code search for agents that uses 98% fe... by esafranchik on Sunday May 17, 2026 @02:01PM (#48171416)

Wouldn't NDCG/token results vary wildly depending on the agent's query and the number of returned items?

e.g. agents often run `grep -m 5 "QUERY"` with different queries, instead of one big grep for all items.

[ Parent | on HN ]

Re:Show HN: Semble – Code search for agents that uses 98% fe... by stephantul on Sunday May 17, 2026 @02:25PM (#48171711)

The same holds for semble: the agent can fire off many different semble queries with different k/parameters.

I guess the point we’re trying to make is that you need fewer semble queries to achieve the same outcome, compared to grep+readfile calls.

[ Parent | on HN ]