aszen

Because subscription plans are cheaper, only enterprises paying per tok pricing should be freaking out

About jev being a black box and the potential for bias, I think it boils down to what questions you are asking the model.

Broad questions like Is this resume good / score this city will ofcourse be biased but I think jev encourages more granular focused questions like Score this candidates Python experience / Rate this city for its food which then allows you to introduce your own biases in which questions you ask and how you combine their answers.

In this way I think jev like models can be easier to reason about for critical decisions.

Their zdr is not a concrete promise though, there's no way to verify that the provider is not storing the logs

I was thinking the same, obvious suspicion is they benchmaxed it on older bench.

Because stronger models are harder to jailbreak from the paper it says haiku was easily fooled into giving us thinking contents

Won't surprise me if the llm just calls sleep after it's convinced it knows all

And does it actually use it

Is your plugin just skills or does it have custom agents?

My own experience is that Claude will use your skills but will ignore your agents or custom search tools.

Claude code already fans out and sandboxes context by calling sub agents so I'm not sure this approach brings much benefit there. A complex search strategy only makes sense if the search is slow and compute intensive.

Coding agents prefer to do iterative search, I have yet to see them create a complex search script. They try different search cmds in parallel, evaluate their results and then refine or dive deeper.

This approach usually works great but I can see many use cases where a smarter search strategy may make sense especially to optimize context.

By slowing down engineers with ai agents adding multiple code reviews on top. Also encouraging engineers to engage in manual testing themselves to better understand the product.

Claude code is not infra, the model is the infra. They changed settings to make their models faster and probably cheaper to run too. Honestly with adaptive thinking it no longer matters what model it is if you can dynamically make it do less or more work.

Same here. Reviewing gets harder too and multi tasking kills any kind of productivity if you need to review the code then.

My approach these days is to do one change at a time, until I can fully merge it with confidence.

This is quite interesting, will try it. I kind of expect this to be done continuously as the code base changes.

This article doesn't mention the moat of data gathering, frontier AI labs have a huge advantage in curating proprietary datasets from actual usage of their platforms.

This in turn allows them to optimize their models for the long tail of tasks that open weight models can't compete with.

Another factor is that pure intelligence isn't enough, how the model communicates is a huge plus. An enterprise used to talking to Claude all day won't be easy to switch to another model

Seems like you are testing llms genric abilities rather than your actual agent logic.

Llms are like vendor code you don't need to test them yourself people already created benchmarks for that.

If you buy real handcrafted scarves they are both thinner and warmer than anything factory made bcz of their choice of pashmina wool.

So the new implementation always operates at the line level, replacing one or more lines. That's not ideal for some refactorings like rename where search and replace is faster.

Edit

Checking ohmypi The model has access to str replace too so this is just a edit till

I bet writing the code directly could have been even faster, llms aren't magically fast

I wonder why we are even storing secrets in .env files in plain text

How? You don't know what the llm was trained on and don't know if it has any bias. Imo llms are a disaster for knowledge work because they act like a black box.

Stupid question but why not use a local sandbox for yolo mode instead of a remote machine.

Is there a similar service that runs locally?

Agreed it probably contributes to the model improving for all agents but crucially it is verifiably better against their own agent. So they get a good feedback loop to improve both

They nailed down the UX I would say and the models themselves are a lot better even outside of CC

Yeah that's one example, but I suspect they train the model on entire sequences of tool calls, so unless you prompt the model exactly as them you won't get the same results.

There's a reason they won the agent race, their models are trained to use their own tools.

Seems odd to not mention other semantic standards that standardize resource operations like pagination, sorting etc.

Jsonld, json:api

The most imp part is editing code, to do that reliably Claude models are trained on their own str replace tool schema I think. Models find it hard to modify existing code, they also can't just rewrite whole files bcz that's expensive and doesn't scale.

Can you elaborate on this?

I don't think so, model improvements far outweigh any harness or tooling.

Look at https://github.com/SWE-agent/mini-swe-agent for proof