News for hackers. Stuff that matters.
Re:The Demonization of DeepSeek: How NIST Turned Open Scienc... by dns_snek on Sunday October 05, 2025 @02:48PM (#45484112)
Like generating vulnerable code given a specific prompt/context.

I also don't think it's just China, the US will absolutely order American providers to do the same. It's a perfect access point for installing backdoors into foreign systems.

Re:The Demonization of DeepSeek: How NIST Turned Open Scienc... by flir on Sunday October 05, 2025 @04:01PM (#45484714)
> Like generating vulnerable code given a specific prompt/context.

That's easy (well, possible) to detect. I'd go the opposite way - sift the code that is submitted to identify espionage targets. One example: if someone submits a piece of commercial code that's got a vulnerability, you can target previous versions of that codebase.

I'd be amazed if that wasn't happening already.

Re:The Demonization of DeepSeek: How NIST Turned Open Scienc... by Imustaskforhelp on Sunday October 05, 2025 @04:09PM (#45484775)
The thing with chinese models for the most part is that they are open weights so it depends on if somebody is using their api or not.

Sure, maybe something like this can happen if you use the deepseek api directly which could have chinese servers but that is a really long strech but to give the benefit of doubt, maybe

but your point becomes moot if somebody is hosting their own models. I have heard glm 4.6 is really good comparable to sonnet and can definitely be used as a cheaper model for some stuff, currently I think that the best way might be to use something like claude 4 or gpt 5 codex or something to generate a detailed plan and then execute it using the glm 4.6 model preferably by using american datacenter providers if you are worried about chinese models without really worrying about atleast this tangent and getting things done at a cheaper cost too

Re:The Demonization of DeepSeek: How NIST Turned Open Scienc... by XorNot on Sunday October 05, 2025 @06:16PM (#45485795)
I think "open weights" is giving far too much providence to the idea that it means that how they work or have been trained is easily inspectable.

We can barely comprehend binary firmware blobs, it's an area of active research to even figure out how LLMs are working.

Re:The Demonization of DeepSeek: How NIST Turned Open Scienc... by Imustaskforhelp on Sunday October 05, 2025 @07:51PM (#45486303)
Agreed. I am more excited about completely open source models like how OlMoe does.

Atleast then things could be audited or if I as a nation lets say am worried about that they might make my software more vulnerable or something then I as a nation or any corporation as well really could also pay to audit or independently audit as well.

I hope that things like glm 4.6 or any AI model could be released open source. There was an AI model recently which completley dropped open source and its whole data was like 70Trillion or something and it became the largest open source model iirc.

Re:The Demonization of DeepSeek: How NIST Turned Open Scienc... by nylonstrung on Monday October 06, 2025 @04:48AM (#45489161)
A backdoor would still be highly auditable in a number of ways even if inspecting the weights isn't viable.

There's no possibility for obfuscation or remote execution like other attack vectors

Re:The Demonization of DeepSeek: How NIST Turned Open Scienc... by flir on Monday October 06, 2025 @05:50AM (#45489587)
I was actually thinking of the American models ;)
Re:The Demonization of DeepSeek: How NIST Turned Open Scienc... by Hizonner on Sunday October 05, 2025 @03:17PM (#45484359)
Up until recently, I would have reminded you that the US government (admittedly unlike the Chinese government) has no legal authority to order anybody to do anything like that. Not only that, but if it asked, it'd be well advised to ask nicely, because it also has no legal authority to demand that anybody keep such a request secret. And no, evil as it is, the "National Security Letter" power doesn't in fact cover anything like that.

Now I'm not sure legality is on-topic any more.

Re:The Demonization of DeepSeek: How NIST Turned Open Scienc... by whatthesimp on Sunday October 05, 2025 @03:26PM (#45484435)
> Up until recently, I would have reminded you that the US government (admittedly unlike the Chinese government) has no legal authority to order anybody to do anything like that.

I'm not sure how closely you've been following, but the US government has a long history of doing things they don't have legal authority to do.

Re:The Demonization of DeepSeek: How NIST Turned Open Scienc... by Ekaros on Sunday October 05, 2025 @04:04PM (#45484727)
Why would you need legal authority when you have whole host of legal tools you can use. Making life a difficult for anyone or any company is simple enough. Just by state finally doing their job properly for example.
Re:The Demonization of DeepSeek: How NIST Turned Open Scienc... by grafmax on Sunday October 05, 2025 @03:25PM (#45484426)
It really does seem like we’re simply supposed to root for one authoritarian government over another.
Re:The Demonization of DeepSeek: How NIST Turned Open Scienc... by nathan_douglas on Sunday October 05, 2025 @09:19PM (#45486751)
Oceania, Eastasia, or Eurasia. Pick one :)
Re:The Demonization of DeepSeek: How NIST Turned Open Scienc... by nylonstrung on Monday October 06, 2025 @04:48AM (#45489163)
My surveillance state is better than your surveillance state!
Re:The Demonization of DeepSeek: How NIST Turned Open Scienc... by hobs on Sunday October 05, 2025 @03:43PM (#45484564)
It doesn't really matter when you have stuff like Quantum Intercept(iirc) where you can just respond faster to a browser request than the originator - inject the code yourself because its just an api request these days.
Re:The Demonization of DeepSeek: How NIST Turned Open Scienc... by nylonstrung on Monday October 06, 2025 @04:41AM (#45489100)
Why would they do this? Deepseek is a private company not owned by the CCP.

There's zero reason or even technical feasibility for them to skip in backdoor that would be easily detected and destroy their market share

None of the security benchmarks or audits show that any Chinese models write insecure code

Re:The Demonization of DeepSeek: How NIST Turned Open Scienc... by dns_snek on Monday October 06, 2025 @04:51AM (#45489196)
I'm not saying that they do this today, I'm saying that China and US will both leverage that capability when the time and conditions are right and it's naive to think that they wouldn't.

Antrophic have already published a paper on this topic, with the added bonus that the backdoor is trained into the model itself so it doesn't even require your target to be using an attacker-controlled cloud service: https://arxiv.org/abs/2401.05566

> For example, we train models that write secure code when the prompt states that the year is 2023, but insert exploitable code when the stated year is 2024. We find that such backdoor behavior can be made persistent, so that it is not removed by standard safety training techniques, including supervised fine-tuning, reinforcement learning, and adversarial training (eliciting unsafe behavior and then training to remove it).

> The backdoor behavior is most persistent in the largest models and in models trained to produce chain-of-thought reasoning about deceiving the training process, with the persistence remaining even when the chain-of-thought is distilled away.

> Furthermore, rather than removing backdoors, we find that adversarial training can teach models to better recognize their backdoor triggers, effectively hiding the unsafe behavior. Our results suggest that, once a model exhibits deceptive behavior, standard techniques could fail to remove such deception and create a false impression of safety.

Re:The Demonization of DeepSeek: How NIST Turned Open Scienc... by FooBarWidget on Monday October 06, 2025 @07:16AM (#45490123)
And how would an open source model ever find out that it's being used by an adversarial country?
Re:The Demonization of DeepSeek: How NIST Turned Open Scienc... by rurp on Monday October 06, 2025 @11:26AM (#45492423)
Companies in China have no intrinsic right to operate in ways that displease the ruling party. If the CCP feels strongly that a company there should or shouldn't do something the company managers will comply or be thrown in jail.