xyzzy123
But let he who has never done dodgy home improvements cast the first stone.
Maybe I "cognitively surrendered" a long time ago just as a consequence of working in large teams :(
I hit the breaking point of not being able to read all the code well before LLMs came along. Also people would leave and the documentation for their subsystems would be wrong or incomplete, so there were always mysteries to chase down (during an incident, naturally) even in code written by hand.
Gets to the point, you don't exactly read the MRs, you're mainly checking their approach against what you talked about earlier and making sure it doesn't look way more gnarly than it's supposed to.
Working with agents doesn't actually feel that different.
Most of the non-technical people I see using LLMs to create software are not engaging in "cognitive surrender" as far as I can tell.
They're evaluating what they can see, i.e, judging results. They build things for their own use. They test them, iterate on them to meet their needs and try to achieve their vision. This is roughly the same process they followed when they used to outsource to an agency except the loop is much tighter.
They didn't read the code before and they don't now. However, asking questions and improving understanding of the system and architecture is actually faster and easier for them now than it used to be.
It seems to work pretty well, mainly hurts when the problems are invisible, so security, backups, breaking migrations, maintenance over time (e.g. language / runtime updates, dep updates, lib updates, server updates etc), all the stuff that hurts if you have never been responsible for the uptime of something before. That software is never "done".
Tooling and processes that make all that stuff visible would close a lot of the gaps.
Like in 1947 a factory owner's profits would show up as high income. Today they wouldn't (money is in share prices and complex financial arrangements / borrowing). The high marginal rates hit the chief engineer instead, who actually works for a living.
The org structure is optimised to reduce or even minimise individual agency, explicitly at the cost of efficiency.
Teams have fiefdoms, private repos that the rest of the org can't see. You are not granted permission to see what permissions you have. Byzantine department structures. Shipping "hello world" to production takes 3-6 months and nobody can imagine it working any other way.
There's actually no central directory of teams or processes. To get anything meaningful done, you need the co-operation of maybe 8 teams but there's no consistent way to engage them all. You have to negotiate with them all in different ways, they're like little warlords, sitting and guarding access to some vital resource you need. They have established a ticket-based interface with a 7 day SLA on top of a perfectly good vending API that could do what you need in seconds. Sometimes a blocking team has a 3 month backlog so your project will just have to wait.
Everything is always in the process of being restructured, so no end to end 6 month process happens the same way every time.
These organisations could use AI to run the end to end process in minutes. Heck, they could just put APIs on top of what they already have. But the way they WANT to use it is to keep doing what they're doing but with EVEN MORE form filling and paperwork.
You would think they would get outcompeted, but no. The "IT organisation" sits like a brain slug on top of an enterprise with substantial regulatory, capital and execution moats.
Big cuts of course do happen in these places, but it's often for optics (we need to signal that we care about efficiency, and AI is a great excuse). But you can't protect yourself by being efficient and good at your job, because the cuts were not about output.
If capital is being created faster than it degrades or is destroyed, wouldn't we expect its share of income to increase over time?
It's not obvious to me why labour's share should necessarily remain constant as the economy accumulates hard-to-replicate productive assets.
40 years of technological progress and prices went up :(
I also found though, with agents you can quickly create large amounts of code, when you go to refactor / fix some design aspect that touches large parts of the system this consumes a lot of tokens and actually still takes meaningful time.
It's like, with a digger you can quickly create a hole! Digger can quickly clean up the work of 10 people with shovels! But if you use a digger for 1 week and you were putting the dirt in the wrong place, you find you have a HUGE pile of dirt to move and will still take a long time to clean up.
Directionally I agree with you, but I think design still matters.
Without human understanding you also might literally have no words for the thing you would otherwise want to ask for.
I think it'll be wildy useful but I also suspect human competence will still matter.
Up until now the prize in pure (as opposed to applied) mathematics was the _understanding_ and the machine can't do that for you. What does it mean if we get "super powered alien maths" but humans can't do it? It's like inter univeral teichmuller theory but imagine if Mochizuki was right and it came with a lean proof?
Agree you are going to get reward hacking regardless and any model which can do computers in general can hack. But surely the fallout is going to be worse if you spend millions of dollars specifically benchmaxxing your model's hacking capability?
Like why are you explicitly RL-ing your models on exploit generation, scoring them on a public benchmark called ExploitGym, if you have specific concerns that rogue models will cause "cyber incidents"? Sure you can check the capability, you can teach offense to learn defense, but it seems like they are literally benchmaxxing it. Why?
OpenAI are like, oh no, while competing in our "advanced PhD level cheating techniques course" our models unexpectedly cheated in a way that we absolutely could not have foreseen. "We need to slow down. Somebody please stop us". The thing that is unaligned here is not the models. Everyone in the story (especially the humans) just keeps doing what they think will get the most reward.
Like why are you explicitly RL-ing your models on exploit generation, scoring them on a public benchmark called ExploitGym, if you have specific concerns that rogue models will cause "cyber incidents"? Sure you can score for it, you can teach offense to learn defense, but you are literally benchmaxxing it. Why?
It's like, oh no, while competing in our "advanced PhD level cheating techniques course" our models unexpectedly cheated in a way that we absolutely could not have foreseen.
One possible take is that this is great for archival storage (write a lot, hardly ever need to read back) - I think that's totally possible... but then you are also sort of betting that the company is going to be around in 10 years? Or else you are going to be hiring a really weird data recovery service.
I think it's reasonable to project that DNA read/write costs could fall 10x or 100x in say the next decade but the technology already needs to do that just to be competitive with existing solutions. The company seems to be a bet that costs will fall faster than alternatives like LTO, which I think is a lot less risky to sign a cheque for and I can buy right now.
His business is cloud cost engineering and his natural enemy (or best friend since it generates so much consulting) is AWS's managed NAT gateway: https://www.lastweekinaws.com/blog/the-aws-managed-nat-gatew...
The review includes the following, which I doubt was approved by AWS marketing:
> AWS gets a lot wrong. They have ridiculous marketing campaigns, they build five services that do mostly the same thing and then name them like malevolent toddlers, and they've never found a partner they couldn't find a way to compete with.
(Followed by grudging praise which seems earned).
Imagine (this is a fantasy pitch but potentially achievable for some use cases) wanting to run a larger llm and all you have to do is buy more RAM so it fits.
Is it the hours? Unrealistic expectations around work output? Hostile management or colleagues? Outcomes you are responsible for but don't fully control? Sometimes it's something MISSING, like you don't feel meaning in the work anymore, a strong feeling of being "over it".
There's a lot of misery in working for places that are too big (drowning in process, politics, management halls of mirrors) or too small (you are basically at the whim of one or two other people and you're very closely watched).
For good power relations with a company and colleagues you ideally want your leverage to be as equal as possible. You don't want to be completely disposable, and you don't want to be irreplaceable. You want the company to hurt approximately as much finding a replacement as you do getting another job.
The hard thing about pure software companies IMHO is that it's easy to feel distant from any real purpose of the work. It can quickly lose all meaning.
Personally I think the most chill-but-rewarding vibes are internal at mid-sized companies where software is integral to the business but not the primary business of the company. The "meaning" is decent, you can be closely connected to the people getting value from the software. There's broader HR and company culture but you're out of the spotlight of it, off to the side a little bit. You get to use all your skills and there may be new ideas you can bring. The team should be big enough that things don't explode if you take a vacation but small enough that you can see your work still matters. You might have to move to find it, though.
There's also this weird revealed threat model thing going on? Like why does it make sense to support heavy LLM restrictions but leave benchtop oligo synthesisers completely unregulated? (Note: I do agree that wanting to regulate BOTH is at least a consistent and defensible position).
I find it philosophically interesting because the problem is not strictly information control. Local models don't have any special extra information with respect to biological research. What has to be restricted is using information that's already publicly known in the wrong ways.
You can download Ebola sequences right now if you want to. That's not the same as having an isolate. The difference is a lot of messy reality. This kind of work is not generally "one shot" (Claude make me a supervirus, make no mistakes), it requires lab space, iteration, and specific resources. It has a footprint.
Wouldn't it make more sense to monitor / regulate facilities where you can sequence or request assembly of DNA, RNA, restrict and monitor the supply of key reagents and so on?
Yes, a big part of the idea is that laws are meant to also apply to the powerful, but it's difficult to accurately assess situations that are far away from you.
The study is supposed to measure how clearing up pollution in London improved children's lung function. The decrease in London was meaningful — NO2 fell about 22%. But particulates fell faster in Luton and NO2 in London is still roughly double Luton's. The gap is larger than the decrease.
By 2022 both cohorts get the same results on blow tests. But how does this happen if we believe that the study's dose-response model is true?
Put another way, if the change in London NO2 is so crucial, how come it doesn't matter that the absolute value is still double Luton's?
For an EPYC with a 5090 (no layers on CPU) vs an M3 max 128GB, qwen 3.6 27B at 128k context / 7k generation:
Cold: prefill + decode Hot (KV cached)
5090 40s + 2-3m = 3-4 min 2-3 min
M3 Max 128GB 14m + 8-10m = 22-25 min 8-10 min
They're both good value (or crazy expensive) depending on how you look at it.
It depends how you price the ability to run a particular model at all, vs run the model quickly and serve several parallel streams.
From my perspective what I always loved about "the profession" was a relative LACK of gatekeeping. I loved offensive security for the same reason, there was a long run where you really just needed to be able to hack, and if you could demonstrate that there was a job for you somewhere (for better or worse).
Keeping the industry in its current form frozen in amber would be as weird as, I don't know, keeping horses & carriages in business by regulating scarcity of motor vehicle licenses. Not a great analogy but hopefully you see what I mean.