News for hackers. Stuff that matters.
Re:DeepSeek V4 Pro 0813 by freakynit on Wednesday August 12, 2026 @02:18PM (#49276546)
Just tested through openrouter.. gave exactly same task.. the task was to scan existing repo, and generate a single docker-compose file to deploy behind a caddy server, where certain port ranges are already used, the service demands widlcard certificates to be provisioned from outside, and postgre needs to be built-in one...

Tested this model, and gpt-5.6-terra-high.

Results: this one had few issues. terra: none.

These results are consistent with my past observations with the latest flash version as well. What benchmarks say, vs what I've been observing are different.

They are good till the project is simple... not anymore.

Re:DeepSeek V4 Pro 0813 by ApolloFortyNine on Wednesday August 12, 2026 @04:05PM (#49277874)
I use deepseek flash to do exactly this. Git repo (which I usually have it build from scratch) -> build docker image -> deploy to server with komodo/caddy-docker proxy.

Works great, regularly one shot applications. I often make changes to the application after its deployed (to be fair, my prompts are usually quite laxidasical, just 'build x, use /deploy-to-komodo) but the deployment works great.

I did make a skill, but if your doing anything repeatedly you should as well.

Opencode, but any harness I'd think would work similar.

Re:DeepSeek V4 Pro 0813 by derangedHorse on Wednesday August 12, 2026 @02:39PM (#49276790)
Terra has not been able to do any of the technical tasks I've asked of it correctly. I'm surprised others get use out of it. Anything below Sol high tends to give me mostly unreliable results. I'm using codex as my main harness but maybe it performs better with a different one.
Re:DeepSeek V4 Pro 0813 by mixedCase on Wednesday August 12, 2026 @03:48PM (#49277659)
With Pi as a harness I've been using OpenAI models as a worker with an Opus 5 (in Claude Code) planner. I've only had a few issues with Terra High/Medium and absolutely none with Sol Medium+ on a fairly complex Rust project that targets Linux, Mac, Windows and Web, with plenty of nasty FFI, VMs, remotely debugging systems, among some other things within a monorepo.

I think the key is to give them a nice assortment of self-verification tools, an AGENTS.md or reference document that they're encouraged to routinely check, and asking the planner to be thorough with the ACs but give the model some space.

The planner routinely finds issues with the worker's output, but that's what it is for.

Re:DeepSeek V4 Pro 0813 by sejje on Wednesday August 12, 2026 @05:32PM (#49278868)
Plan with sol-med, implement with luna-high. Rarely a problem.
Re:DeepSeek V4 Pro 0813 by ericfr11 on Wednesday August 12, 2026 @06:23PM (#49279402)
Same for me, with Claude Opus/Sonnet. All the models are almost equivalent if well steered
Re:DeepSeek V4 Pro 0813 by ericfr11 on Wednesday August 12, 2026 @06:25PM (#49279428)
Harness is the key. I built my own to "talk" our institutional knowledge and it's working great
Re:DeepSeek V4 Pro 0813 by zeven7 on Wednesday August 12, 2026 @04:16PM (#49277982)
I bounce between Sol high/medium and Luna max. I don't know why you'd use anything between Luna max and Sol medium. Luna is so extremely cheap and cranked up to max it does anything I'd want Terra to do for a fraction of the cost. What is Terra for?
Re:DeepSeek V4 Pro 0813 by Juvination on Wednesday August 12, 2026 @04:48PM (#49278344)
One thing I've really noticed with Luna Max is its speed. I've got a review script setup on a custom Pi extension. Luna finds some issues/some false positives, while Sol finds issues but disregards false positives. The biggest thing is Sol finishes in about half the time.
Re:DeepSeek V4 Pro 0813 by freakynit on Wednesday August 12, 2026 @02:41PM (#49276810)
Depends on project complexity. For one of my more complex projects, I exclusively use sol-high ... nothing below that works correctly.

For this however, a comparatively much simpler task, tarra-high works fine.

Re:DeepSeek V4 Pro 0813 by Foobar8568 on Wednesday August 12, 2026 @03:02PM (#49277073)
Right now, sol-xhigh is my favorite model. I feel that Opus 5 is dumber than 4.8. Fable is too expensive to do anything (limit of $50, started a prompt at $25, ended up at $75, is bullshit, but at least it's "free credits").

DeepSeek is okay for random API-based stuff, as it's cheap.

Local open models running on a 5090 are hit or miss. I feel that most GGUFs/quants are awful...

Re:DeepSeek V4 Pro 0813 by miohtama on Wednesday August 12, 2026 @05:23PM (#49278750)
Opus 5 degrades to word salad.

I wonder if it is because of watermarking.

Re:DeepSeek V4 Pro 0813 by SwellJoe on Wednesday August 12, 2026 @06:39PM (#49279550)
Opus 5 doesn't really even speak coherent English. I'm not sure what's going on, but it can't explain anything. It still does an excellent job with code and writing tests and code review and creating and completing a plan, and it seems to be able to understand English instructions, but it sure as hell can't explain what it did or how to use the code it wrote.

That was true before they announced the watermarking, I'd already started to back off of using Opus as much because I like to understand what the model is doing and have it write documentation I can use to reproduce its results, but maybe watermarking was already in there unannounced.

Re:DeepSeek V4 Pro 0813 by sscaryterry on Thursday August 13, 2026 @04:48AM (#49283261)
This 100%. The incoherent drivel has literally made me move away from CC completely.
Re:DeepSeek V4 Pro 0813 by mlrtime on Thursday August 13, 2026 @07:03AM (#49284205)
Agreed, I'll get a terminal full of text and I just reply "I don't understand this"

And it retypes it for a human, I'm doing this more and more lately.

Re:DeepSeek V4 Pro 0813 by pulkitsh1234 on Thursday August 13, 2026 @08:08AM (#49284763)
Ah, so it's not only me :-D

I think it could be the watermarking, but at this point they might be deliberately complicating the prose so that we ask clarifying questions and that leads to more token spend.

Re:DeepSeek V4 Pro 0813 by ericfr11 on Wednesday August 12, 2026 @06:21PM (#49279387)
I am still on Opus 4.8, with a custom built harness and it works very well even on multi-repos, across stack, deep changes. I also have a very solid test suite which is helping the coding agent a lot
Re:DeepSeek V4 Pro 0813 by bob1029 on Wednesday August 12, 2026 @03:34PM (#49277500)
I am seeing essentially deterministic results with Terra running a custom browser automation agent across >100 interaction events.

The harness is everything. If I just threw something like Codex at this and said "good luck" I wouldn't make it beyond 5-10 interactions. I tried that already. Carefully designing the views and tools over the environment is where you can go from 50% to 99.9999%.

Re:DeepSeek V4 Pro 0813 by tanishqkanc on Wednesday August 12, 2026 @04:12PM (#49277948)
Curious about what you found. I agree harness for browser automation is vital - I work on https://libretto.sh
Re:DeepSeek V4 Pro 0813 by bob1029 on Wednesday August 12, 2026 @07:15PM (#49279852)
Hand-crafted adapters that sit between playwright primitives and the agent loop are the secret sauce. The goal is to insulate the agent from the raw DOM without any loss in fidelity regarding the logical business information and available actions.
Re:DeepSeek V4 Pro 0813 by drewnick on Thursday August 13, 2026 @12:08AM (#49281708)
+1 I have found extremely reliable systems require a mix of deterministic "adapters" is a good word for agents to actually get through the workflows I've created. I'm still amazed that it can work with both those and some "intuition" to bend the rules around the adapters if prompted.
Re:DeepSeek V4 Pro 0813 by cyanydeez on Wednesday August 12, 2026 @02:59PM (#49277039)
the breadth and width of the universe of oneshot challenges are all arbitrary. It's unsurprising different workflows oneshot better than others.

All the more reason to favor local models under your control, as once you find that sweet spot model, no one can change it, upgrade it, align it, take it down or otherwise harm the time investment you made it making it your own.

I can't really believe no one understands, after decades, how valueable a rock solid development environment is.

Re:DeepSeek V4 Pro 0813 by Phemist on Wednesday August 12, 2026 @04:14PM (#49277965)
Exactly! I am not opposed to cloud-based models, but I do only stick to open-weight models because I know I can move my whole stack to local (given enough hardware) and continue development without any of the LLM interaction contracts being broken.

I would like to see some development where proof of authenticity certs are generated alongside the actual output of the model. Prove to me (or at least claim to me liable to breach of contract) that this output was generated by FP8 DeepSeek V4 Pro 0813. Not some cheaper quantization of the model.

Re:DeepSeek V4 Pro 0813 by f6v on Thursday August 13, 2026 @07:32AM (#49284441)
But I don’t want to manage a local model…
Re:DeepSeek V4 Pro 0813 by cyanydeez on Thursday August 13, 2026 @09:07AM (#49285352)
Likr sll infrastructure, once its setup and your flow is set, the only thingin the way is FOMO.
Re:DeepSeek V4 Pro 0813 by Art9681 on Wednesday August 12, 2026 @05:31PM (#49278847)
Terra is great. It's wild how different our experiences are.

Install the Superpowers plugin.

Behold.

Re:DeepSeek V4 Pro 0813 by MuteXR on Thursday August 13, 2026 @05:34AM (#49283567)
In my experience, Superpowers has begun to massively slow down the capable models at this point. The skills they add are incredibly bloated and just get you worse results nowadays, tbh.
Re:DeepSeek V4 Pro 0813 by agrippanux on Thursday August 13, 2026 @07:15AM (#49284310)
I loved Superpowers and evangelized it heavily.

This week I removed it because it now gets in the way of the frontier models.

Re:DeepSeek V4 Pro 0813 by npn on Wednesday August 12, 2026 @02:32PM (#49276732)
wait for Deepseek Harness (yes it is the official name) release then try again.

for your kind of task, harness tools matter.

Re:DeepSeek V4 Pro 0813 by gkbrk on Wednesday August 12, 2026 @02:40PM (#49276799)
If the model cannot figure out simple and ubiquitous tools, how is it supposed to figure out complex problems? All of the good models basically work with any harness, including giving them a single "shell command" tool. They can just figure things out.
Re:DeepSeek V4 Pro 0813 by hadlock on Wednesday August 12, 2026 @02:45PM (#49276851)
When it comes to quality of outcome, since at least Feburary, the harness has almost equal, if not more weight than the model itself. It's no longer "which model is the best?" it's "which model + harness is the best?"

I get drastically different tool call failure rates using Claude SDK vs OpenCode using Qwen 3.6 models

Re:DeepSeek V4 Pro 0813 by KronisLV on Wednesday August 12, 2026 @02:51PM (#49276925)
> the harness has almost equal, if not more weight than the model itself

This feels like a horrible failing of the models to generalize, then - both basic and intermediate tasks should be possible to do with Claude Code, OpenCode, Pi, ZCode, Kimi Code, Dirac and tbh any other mainstream or even slightly niche harness. Not doubting the claim itself, there's a reason why good benchmarks include the harness.

Re:DeepSeek V4 Pro 0813 by dominotw on Wednesday August 12, 2026 @05:20PM (#49278713)
i think thats BS that harness has equal weight. most of intellegice is still coming from training data not from RL. so how is 'coevolved harness' equal weight.
Re:DeepSeek V4 Pro 0813 by disgruntledphd2 on Thursday August 13, 2026 @06:25AM (#49283909)
> most of intellegice is still coming from training data not from RL.

For coding specifically, I'm not sure this is still true. Given the heavy use of RL to improve coding performance, I'd expect the harness to be important as it defines what tools the model is rewarded for using.

Re:DeepSeek V4 Pro 0813 by dominotw on Thursday August 13, 2026 @08:59AM (#49285256)
i belive they looked at the traces and they were still legible ( RL traces should be gibberish)
Re:DeepSeek V4 Pro 0813 by JLO64 on Wednesday August 12, 2026 @03:23PM (#49277349)
It's worth nothing that recent Claude models seem to have gotten worse at tool calling outside of Claude Code and the SDK: https://lucumr.pocoo.org/2026/7/4/better-models-worse-tools/
Re:DeepSeek V4 Pro 0813 by davidlt on Wednesday August 12, 2026 @02:53PM (#49276953)
I just wanted to emphasize this. Harness is a big part of how things perform thus usually it's harness + model co-design that's important.
Re:DeepSeek V4 Pro 0813 by azinman2 on Wednesday August 12, 2026 @02:48PM (#49276887)
Which works better for you?
Re:DeepSeek V4 Pro 0813 by HDBaseT on Wednesday August 12, 2026 @06:28PM (#49279455)
Yeah this is a complete lie.

You can use effectively any harness and get good results. Harnesses are mostly placebo.

Re:DeepSeek V4 Pro 0813 by scrlk on Wednesday August 12, 2026 @07:10PM (#49279825)
With Opus 4.7, there's a 10 point improvement in the AA coding agent index when you swap out Claude Code for OpenCode: https://artificialanalysis.ai/agents/coding-agents#harness-c...

To put that in perspective, the difference between GPT-5.6 Sol Max and 5.6 Luna Max is 8 points. That's a lot of extra performance that you can get for free just by using the best harness.

Re:DeepSeek V4 Pro 0813 by RideOnTime22 on Wednesday August 12, 2026 @06:47PM (#49279629)
Every other week it's a new "X didn't matter, until Y date" without any hard quantitative claims.

It's crazy how over the past years a field originating from math ends up succumbing to subjective feels.

Re:DeepSeek V4 Pro 0813 by hadlock on Wednesday August 12, 2026 @09:08PM (#49280629)
It's a forum, not an engineering conference, but here is a demonstrated 10 point difference between two top harnesses anyways: https://artificialanalysis.ai/agents/coding-agents#harness-c...
Re:DeepSeek V4 Pro 0813 by npn on Wednesday August 12, 2026 @02:52PM (#49276952)
I don't think so. there is a lot of tools with similar usage, some harness even bring their own internal tools for accurately manipulation.

also, even if some models claim that they have full 1M context window, some only work effective with the head or tail of the window, a proper harness tool will know about the limitation of the model and act accordingly.

then also the output format, the tool calling syntax, the quirks and gotchas of each model.

it is not simple as just throwing everything at the model, especially when your project has hundred of files or so.

Re:DeepSeek V4 Pro 0813 by derefr on Wednesday August 12, 2026 @03:01PM (#49277068)
Because complex problems can be decomposed (a skill in itself) into easy parts and hard parts; and the hard parts are almost always bottlenecked on understanding concepts and principles (i.e. things that are either in a model's weights, or not), not on having certain facts available. Models can solve complex problems insofar as they can decompose those problems, and have learned the concepts and principles relevant to approaching the hard parts of those problems.

Whereas tool-use isn't a capability problem, but a context problem: the thing that makes models fail by default is that they have no idea, when first summoned out of the aether, what kind of conversation they're having, who it's with, what that person is trying to do, what tools they have available, and how those tools can be invoked.

Think of the difference between how you'd respond to a casual programming question asked by a person sitting next to you on a flight, vs. a programming question asked of you by someone you're pair-programming with with your IDE open in front of you. Now imagine waking up blind and deaf and needing to discern which of the two situations you're in. LLMs know how to approach both of these problem-contexts (and more besides), but they need to be given context to know which problem-context they're in (and everything else about that problem-context: which IDE they're using, which OS it's installed on, what other tools are installed+accessible, etc.)

And before you say "but why can't they just experiment to figure these things out" — if you think about it, knowing how to interface with a shell and an IDE are bootstrapping requirements for any kind of experimentation, in about the same way that "knowing how to open your eyes and move your head" is a bootstrap requirement for a human gaining information about the world around them. These capabilities are necessary to explore the world to "discover" and "probe" other capabilities.

---

Also, a lot of the work LLMs do "needs" (i.e. is heavily improved by the use of) some kind of structured scratchpad, that they have been trained to manipulate and "look at" through tool-use. Even for a human who could accurately visualize a canvas based on a coordinate system, you still wouldn't expect said human to succeed at the pelican test if they had to write the SVG entirely in their head and then write it out sequentially with no rewinding to fix mistakes. You'd expect them to ask for at least a whiteboard, if not a text editor, to be able to write and rewrite the SVG XML.

(Really, they'd ideally want to run the SVG and look at it to see how close it is, and optimize that way. I'm not sure if we're letting LLMs do that part in the classical pelican test. It feels like that would vaguely violate the "zero-shot"-ness of the test, though I'm not sure if we're currently considering a conversation to be "zero-shot" if it involves the model iteratively interacting with a third-party system [such that there are repeated model -> system -> model conversation turns] but holding off to responding to the user until they think they've fully solved the problem.)

---

And also, on a lower level, all of these external capabilities are getting exposed to the LLM through MCP. Models can and do understand how to speak MCP itself. But there's no standard for how a given harness's capabilities (e.g. "execute command line in new shell session", "send patch edit command to active tab in IDE", etc) should be modelled to be exposed through MCP, either in their encoding or in their semantics. There's no MCP equivalent of WASM's WASI meta-standard, such that models could learn these specs and "assume by default" that things work like them until told otherwise; and nor are there even open harnesses that LLMs could learn about during training, and through them, learn some de-facto MCP-endpoint specs. Instead, there are mostly just proprietary harnesses, that hide all that info from public access, sharing it only with the LLM during inference, and even then, only at the moment the LLM needs it.

Re:DeepSeek V4 Pro 0813 by segmondy on Wednesday August 12, 2026 @03:59PM (#49277790)
the single shell command is the terminal bench.
Re:DeepSeek V4 Pro 0813 by sheeshkebab on Wednesday August 12, 2026 @02:46PM (#49276868)
This. The same goes for “skills”, skill type “subagents” and other bullshit - powerful models don’t need any of that anymore I noticed.
Re:DeepSeek V4 Pro 0813 by bugglebeetle on Wednesday August 12, 2026 @02:54PM (#49276974)
Re:DeepSeek V4 Pro 0813 by ghm2199 on Wednesday August 12, 2026 @03:35PM (#49277520)
I use pi harness with codex and all the tool calls are custom delegate extensions, I mean ALL(for security checks), i get consistently good results from sol on high and xhigh reasoning. I don't believe harness should matter because its at most just a way to abstract tool calls and maybe the system prompt. Training on the tool calls results should not(and in codex's case does not matter)
Re:DeepSeek V4 Pro 0813 by freakynit on Wednesday August 12, 2026 @02:38PM (#49276783)
I used pi
Re:DeepSeek V4 Pro 0813 by natrys on Wednesday August 12, 2026 @02:55PM (#49276989)
For me, flash 0731 was much better in omp/opencode than in Pi.

Anyway, it might be so that they are rolling out deployment. There haven't been an official announcement post yet (this submission is a link to openrouter). Some people have been saying they are getting results worse than GLM-5.1, that's obviously broken.

Re:DeepSeek V4 Pro 0813 by teravor on Wednesday August 12, 2026 @05:16PM (#49278668)
> Deepseek Harness

I wonder if it's just going to be a rebranded Reasonix (https://github.com/esengine/DeepSeek-Reasonix) as it's the only harness their docs mention and it seems specially built for deepseek models.

Re:DeepSeek V4 Pro 0813 by scrlk on Wednesday August 12, 2026 @02:36PM (#49276768)
What harness are you using? DS V4 is harness sensitive.
Re:DeepSeek V4 Pro 0813 by lousken on Wednesday August 12, 2026 @02:43PM (#49276833)
Are we testing the model or the harness? If benchmarks show certain numbers it should perform as such without it
Re:DeepSeek V4 Pro 0813 by ApolloFortyNine on Wednesday August 12, 2026 @04:08PM (#49277903)
You can't even run a benchmark without a basic harness, of course the harness has some effect.
Re:DeepSeek V4 Pro 0813 by 0xbadcafebee on Wednesday August 12, 2026 @04:47PM (#49278337)
Re:DeepSeek V4 Pro 0813 by freakynit on Wednesday August 12, 2026 @02:37PM (#49276781)
Pi
Re:DeepSeek V4 Pro 0813 by apitman on Wednesday August 12, 2026 @03:38PM (#49277556)
Wait people use terra?
Re:DeepSeek V4 Pro 0813 by miohtama on Wednesday August 12, 2026 @05:26PM (#49278792)
I use mostly Terra. Much better than Opus 5. Much more token mileage.
Re:DeepSeek V4 Pro 0813 by apitman on Wednesday August 12, 2026 @06:25PM (#49279429)
But why? Luna Max is almost the same intelligence as Terra xhigh and way way cheaper. And Terra max is almost the same as Sol high. I just don't really see a place for Terra but slower.
Re:DeepSeek V4 Pro 0813 by miohtama on Thursday August 13, 2026 @02:02AM (#49282243)
This is a good tip. I will try. Thank you!
Re:DeepSeek V4 Pro 0813 by smb06 on Wednesday August 12, 2026 @06:11PM (#49279305)
My company pretty much exclusively uses Sol and Luna
Re:DeepSeek V4 Pro 0813 by shimman on Wednesday August 12, 2026 @02:27PM (#49276660)
I've always wondered if I was using containers wrong because none of them I've ever had to create were complicated. Maybe it's because I choose tools that make local development easy (Go + sqlite + various CLTs) or maybe it's because I never hard to interact with this on the professional side outside of making images for our projects (which still weren't complicated for the reasons above).

LLMs make containers in a pretty workable format for me (still hand tweak the env variables for a sanity check).

How exactly does it struggle here and why does postgres need to be built? Were the needs beyond what you get in a base image?

Re:DeepSeek V4 Pro 0813 by freakynit on Wednesday August 12, 2026 @02:35PM (#49276751)
This was the repo: https://github.com/amalshaji/portr

And this was my gh issue: https://github.com/amalshaji/portr/issues/308

And below was my prompt:

""" give me single docker-compose file that i can run on my server to run current project... you can read README.md , and then, this relevant page: https://docs-custom-reverse-proxy.portr-docs.pages.dev/docs/... ... this was the result of me raising github issue: https://github.com/amalshaji/portr/issues/308 ... you can use gh cli to fetch the details and comments...

i already have a caddy server running on my vps... and i will create wildcard certificates myself using certbot.. the domain name will be helloportr.xyz ... also, ports up to 9019 are already taken...

ask me if anymore info is needed... """

You can try yourself and let me know of what you got.

Re:DeepSeek V4 Pro 0813 by shimman on Wednesday August 12, 2026 @07:01PM (#49279752)
This is definitely beyond my capabilities lol but wow portr is a neat project. Never heard of it before, only the paid services from tailscale/cloudflare.
Re:DeepSeek V4 Pro 0813 by arch-choot on Wednesday August 12, 2026 @11:00PM (#49281315)
I've been using DS4F+Pi with great results, but I think one thing that helps is at the end of my prompt I'll tell it how to verify it, e.g. "Make sure the compose file works by running it locally (use self-signed certs if required)".

The argument could be made that "the model should be smart enough to figure it out" , and maybe DS4 isn't. But with just a bit of steering you can get the correct result for like 1/10th the cost, or even cheaper.

Re:DeepSeek V4 Pro 0813 by tripleee on Wednesday August 12, 2026 @03:10PM (#49277190)
sir you tested a single fucking task
Re:DeepSeek V4 Pro 0813 by yassa9 on Wednesday August 12, 2026 @04:19PM (#49278021)
did you test kimi k3 or qwen 3.8 max on the same task ? or plan to test them ? I respect those genuine users tests other than those benchmarks that models are trained and overfitted to them
Re:DeepSeek V4 Pro 0813 by v3ss0n on Wednesday August 12, 2026 @04:20PM (#49278037)
I do that kind of things all the time with Qwen 3.5 122B. It works well in one shot with Cline or Opencode.

May be your harness problem?

Re:DeepSeek V4 Pro 0813 by amelius on Wednesday August 12, 2026 @04:23PM (#49278067)
I didn't understand your use case, so it could also be the way you write your prompt, I suppose ...
Re:DeepSeek V4 Pro 0813 by celsoneto07 on Wednesday August 12, 2026 @06:03PM (#49279221)
I've been doing pretty heavy stuff with DeepSeek with a good degree of success. The thing is: I don't trust it to go fully autonomous. I check the steps, I steer it. For the pricing, it's worthy. Let's how the price increase is going to change my behavior.
Re:DeepSeek V4 Pro 0813 by shunia_huang on Wednesday August 12, 2026 @10:08PM (#49281048)
The flash model will always use an outdated Treafik version that is not compatible with the newer docker engine, I tried to deploy some personal services with Traefik and everytime it uses this wrong version, and then fixes the version issue in the thinking chain.

I was thinking to switch to Caddy but with your experience I'm gonna stay with Traefik and bare with the version issue...

Re:DeepSeek V4 Pro 0813 by acchow on Thursday August 13, 2026 @02:54AM (#49282547)
I agree. The “frontier level” open weights models benchmark really well but fall behind in real world performance