angry_octet
angry_octet on Hacker News
The responsible next step is to forward evidence of embargo leaks to the coordinators, so everyone can agree to release earlier.
But they didn't observe any exploits in the wild. They may have believed there were exploits, but no evidence of this. Also no evidence that sought to communicate the need to end the embargo earlier. Instead they sought short term commercial advantage.
My advice to their customers is to leave, Chainguard will be the last to know about exploits in future.
Well it's going into the trash compactor along with all the other jobs that are highly vulnerable to automation with LLMs, but at the same time LLMs have spawned a panic buying spree of AI pentesting contracts. It's going to be a wild ride before the dust settles.
I'd like something to pick up clothes from the floor and fold them away. Something to stack and unstack the dish washer. As long as the bot wouldn't accidentally set fire to the house.
I assume all megacorps are minimising their tax. It would be idiotic to assume otherwise.
I think the draft NDAA blocks running Chinese models on American hyperscaler hardware, but it's principally because they were using it to bypass chip export restrictions.
I don't think any Federal agencies are using DeepSeek inference, but many State agencies would be, primarily through third party services. Of more concern is the critical or financially important infrastructure which is enabled by Chinese models. Banking, ecommerce, retail.
Does DeepSeek not maintain an international financial structure to keep international profits offshore until it is convenient for them to be taxed? I can't think of a billion dollar corp that doesn't, but then Jack Ma went to a gulag for a year.
I think K&R (Ken Thompson and Dennis Ritchie) are very close in my brain's feature space. Reference [1] is the article you are referring to. An AI trojan could conceivably insert itself into the model weights / training data on systems it had access to.
Moving the goalposts a fair distance from DeepSeek, but whatever. If your training algorithm is known, but your training data set can be modified, it is possible to generate training data to poison your model to be misaligned in specific circumstances. Poisoned data is not obvious. Good luck determining if misalignment has occurred.
Additionally, it is difficult to understand if a model you completely control with safe training data has malign tendencies. It's an open research question why this happens. There is a good explanation of this by Owain Evans:
https://www.youtube.com/watch?v=KrM5c0vp8s0
https://owainevans.github.io/
Although you can often take losses from a previous year to offset current income. So if you do a huge amount of capital investment you can offset against profits for years.
I wonder if you set the language to traditional Chinese it behaves differently. Russian commercial malware won't detonate if the language is Russian.
Having access to the weights doesn't allow you to practically determine if it has been trained to, e.g. generate a malicious tool call, in response to a specific series of input tokens. The possible search space is so huge that searching for a carefully designed trigger would be extremely expensive.
It is typical that they conflated the website chat interface to DeepSeek-hosted inference with Amazon and other hyperscaler inference, and self-hosted / on-prem inference. Of course they also confused testing vs production, and left grey what is means when another company re-spins an open weight model (e.g. Cloudflare Clef is Alibaba Qwen derived).
They haven't officially banned Qwen, Kimi etc, but the same risks apply, and I think a ban is coming (certainly is in the USA). It is quite possible for any or all of these models to have a Ken Ritchie style self-propagating back door[1] or a simple phone home baked into the weights[2], and possibly triggerable misalignment in specific circumstances caused by poisoned training data.
[1] https://dl.acm.org/doi/10.1145/358198.358210
[2] https://news.ycombinator.com/item?id=50159245
If Facebook had bought github, they would have made automated squatting mirrors for all the packages like xscreensaver.
This is an ahistorical take. Google was big mad, and I think Apple was too.
Or an employee with true loyalty to another entity. Cf the OpenAI employee alleged to have planted a voice bug.
I can't accept that Wikipedia is an authoritative source, but existence of FTL is not the sole determinant. That would make 'The Lion, The Witch and
The Wardrobe' space opera.
Space Opera is any true descendent of Van Vogt's "Voyage of the Space Beagle" or EE Doc Smith Lensman series, Horatio Hornblower but in space, etc.
It requires the space opera quality, not the soap opera style.
Star Trek is in, Star Gate (core, not including their Universe and Atlantis) just jumps around with the Star Gate. It's Sci-Fi but not space opera. Probably a budgetary constraint of episodic TV, but nonetheless.
Dr Who is occasionally on space ships, there are energy weapons, alien creatures, etc. But these are incidental to the plot, and it isn't Space Opera.
Without space ships? Without space?
Without senior developers having a deep understanding of the problems underlying the systems, they'll have little to distinguish themselves from non-skilled management poltroons, who are themselves even more replaceable by agents. But the alignment problem will become even more severe.
This is detail that would have been interesting to read in the original post.
I think that is exactly what it implied. Essentially, that people had described the task the VCS must achieve, the constraints, the issues with git, and Mythos had toiled away like Rumpelstiltskin and produced Scion 0.1.
Anthropic had an urgent problem, which is why they have reverted to a quick hack of centralisation. However there is a more important goal which will swing them back to decentralization and federation: they need isolation.
All the evidence of AI hacking events points to the need for agent source control and collaboration in completely controlled environments, which precludes giving access to the one-repo-to-rule-them-all in an AI-coded VCS.
I think we can safely assume that all the major sponsors from 2025 pulled out. Any one of them could have carried the whole thing.
https://codewithkate.squarespace.com/blog/the-complete-grace...
Tech Giants & Major Players
Amazon - Booth 751
Google - Booth 1246
Meta - Booth 1010
Nvidia - Booth 1039
Disney - Booth 946
Snap - Booth 447
Roblox - Booth 1209
PlayStation - Booth 433
Nintendo - Booth 429
Finance & Trading Firms
Goldman Sachs - (Location TBD)
Morgan Stanley - Booth 633
Jane Street - Booth 618
Citadel - Booth 62
Two Sigma - Booth 1046
DE Shaw - Booth 410
IMC - Booth 1051
Walleye Capital - Booth 1052
Apollo Global Management - Booth 647
Five Rings - Booth 310
Cboe - Booth 952
Banking & Financial Services
Bank of America - Booth 439
Capital One - Booth 831
Bloomberg - Booth 824
Navy Federal - Booth 426
USAA - Booth 614
Vanguard - Booth 819
Insurance & Financial Tech
Prudential - Booth 526
CHUBB - Booth 344
Voya Financial - Booth 1226
NEXT Insurance - Booth 1227
Manulife - Booth 1122
Tech & Software Companies
Intuit - Booth 422
PayPal - Booth 514
Cloudflare - Booth 533
Wayfair - Booth 342
Xero - Booth 343
Adyen - Booth 605
WEX - Booth 1014
Chewy - Booth 1114
Infoblox - Booth 852
InterSystems - Booth 518
Red Hat - Booth 919
DTCC - Booth 418
Consulting & Professional Services
EY - Booth 1228
E-commerce & Entertainment
Audible - Booth 1251
Automotive & Manufacturing
Rivian - Booth 1022
John Deere - (Location TBD)
Crown Equipment Corporation - Booth 1210
Automotive & Manufacturing
Rivian - Booth 1022
John Deere - (Location TBD)
Crown Equipment Corporation - Booth 1210
Real Estate & Corporate Services
CBRE - Booth 341
Lumen - Booth 345
Retail & Food Services
McDonald's - Booth 339
Healthcare & Medical Technology
Tandem Diabetes Care - Booth 313
Northwestern Medicine - Booth 1214
Mortgage & Real Estate Tech
Rocket - Booth 1211
Agreed. You have to give them device-in-the-loop with trace or a decent simulator. Presumably the big EDA companies are working on tooling integration and bespoke models, and they have huge libraries of designs to draw on.
If you have serious attack surface minimisation goals you need to embrace FOGAs. LLM coded unikernels have way too much bloat.
The Zynq UltraScale+ parts pair a hard ARM CPU with FPGA fabric. You can progressively isolate fast path and high security components in logic. LLMs are getting quite good at using synthesis tools.
https://www.amd.com/en/products/system-on-modules/kria/k26/k...