greyskull
What does "mainstream" refer to when we're talking about software development and LLMs? As opposed to "engineers".
Over the last few months, I've been digging into performance problems with a high throughput service that my team owns. I started working on the problems in my own time, put out short and medium term improvements that legitimately avoided operational issues, and started developing an alternate architecture that should meaningfully address the problems for the long term.
I've learned new things and made improvements that probably wouldn't have ever gone in otherwise.
I have colleagues that seem perfectly content to delegate too much to the agents, and it saddens me. It feels like there will be swaths of engineers that didn't train some of the critical thinking skills that I take for granted.
I certainly see it in slack discourse around anything more complicated than a feature implementation. Maybe I'm just cynical. Time will tell, I suppose.
This is tangential: and forgive my ignorance here, but is there an inherent reason why there aren't smaller, focused models from the frontier model providers?
I'm thinking something like a software-specific subset of Opus that is the default for use in Claude Code. Smaller, cheaper to deploy and consume, maybe faster.
How did you land on that model? Hard to tell if I should be a) going to 3.5, b) going to fewer parameters, c) going to a different quantization/variant.
I didn't consider those other flags either, cool.
Are you having good luck with any particular harnesses or other tooling?
> nothing you can run locally, on that machine anyways, is going to compare with Opus
Definitely not expecting that. Just wanted to find a setup that individuals were content with using a coding harness and a model that is usable locally.
What does your setup look like? Model, harness, etc.
Aside: what is your tooling setup? Which harness you're using (if any), what's running the inference and where, what runs in WSL vs Windows, etc.
I struggle to even ask the right questions about the workflow and environment.
Does anyone have a similar experience of having thoroughly used CC/Codex/whatever and also have an analogous self-hosted setup that they're somewhat happy with? I'm struggling a bit.
I have 32GB of DDR5 (seems inadequate nowadays), an AMD 7800X3D, and an RTX 4090. I'm using Windows but I have WSL enabled.
I tried a few combinations of ollama, docker desktop model runner, pi-coding-agent and opencode; and for models, I think I tried a few variants each of Gemma 4, Qwen, GLM-5.1. My "baseline" RAM usage was so high from the handful of regular applications that IIRC it wasn't enough to use the best models; e.g., I couldn't run Gemma4-31B.
Things work okay in a Windows-only setup, though the agent struggled to get file paths correct. I did have some success running pi/opencode in WSL and running ollama and the model via docker desktop.
In terms of actual performance, it was painfully slow compared to the throughput I'm used to from CC, and the tooling didn't feel as good as the CC harness. Admittedly I didn't spend long enough actually using it after fiddling with setup for so long, it was at least a fun experiment.
I _think_ you can get scale to zero on Lambda by deploying a docker container, too.
2) We'd have to define what constitutes low traffic vs any other arbitrary measure, so it's moot to discuss like this; all I said it wasn't high traffic. You could run it for cheaper, but there wasn't much expertise for self-hosting, for example.
3) For all I remember it may have been half that in daily cost. In any case, miniscule compared to engineer time. What was worse was the prior decision to use serverless aurora rds, that dwarfed everything else in AWS cost - I know this is tangentially related, just saying optimizing that a bit more was not the highest priority, we could do it for cheaper.
The last question is salient, and it's possible for OpenNext to break and have to catch up to changes in Next.js, though I believe there's some more direct collaboration. I'd say that's the biggest downside - it's not guaranteed compatibility.
I did a migration recently (comments elsewhere in this post), and I don't recall the specific issue, but I _do_ recall running into at least one scenario where OpenNext had made a decision that impacted - in a way that was visible to me and undesirable - how Next.js functioned. That's not a criticism, there's tradeoffs.
Some of them are, for example, Terraform projects that list the specific infra. I have experience with the SST deployment, whose website unfortunately doesn't do a great job of listing the infra architecture.
Migrating to OpenNext using SST, I think we got the bills for compute and asset serving down to like $15/day or something (granted, we spent expensive engineer time on the migration).
[0] https://nextjs.org/docs/app/api-reference/components/image#s...
As other commenters have touched on - my understanding is the purpose of OpenNext is to package the output artifacts of a Next build in a way that can be deployed to a serverless environment, analogous to how Vercel does it. The supporting projects like SST and the other links in the repo are to take those OpenNext artifacts and deploy them to infrastructure generally in an opinionated way - additionally supporting some of the "extra" features described in the repository.
The last project I was working on was to then migrate from SST to Fargate, as a persistent process (serverful?) deployment was preferable for various reasons. In that scenario, we would just be running the built in server using the Next.js standalone deployment mode (effectively a `node index.js`). We didn't need the extra functionality covered by OpenNext.
I see that the book is incomplete. I didn't know that early access for books was a thing, very neat. It might be pertinent to note in your post that it's still being written, with an estimated release window of Spring 2025.
I'm very much a "consume it when it's ready" person, so I'll keep this on my watch list.
Our Bronco Sport has this (steering wheel vibration) in addition to lane assist. I don't like the latter, so I appreciate that I can enable/disable the two modes independently.
If you don't want to do that and keep their gateway, I'd expect you can run pi-hole anyway, and if you're not getting the behavior you want (e.g., the gateway seems to be intercepting dns or something), you can try DNS-over-HTTPS.
I thought maybe it's a database optimization, but it could also just be an unfortunate design decision.
Depending on how old the service is, the ownership of the CFN resource may be on the CFN service team (from back when they white-gloved everything) in which case there are massive development bottlenecks (there are campaigns to migrate these to service team ownership) or more often the resource is maintained by the service team itself, in which case the team may not be properly prioritizing CFN support. There can be a whole lot of crap to deal with for releasing a new CFN _resource_, though features within a resource are relatively easy.
On my last team, we did not consider an API feature delivered until it was available in CFN, and we typically turned it around within a couple of weeks of general API availability.