darkwi11ow
darkwi11ow on Hacker News
> No, LLMs are not a classical symbolic system applying explicit rules to symbols.
I do not mean that LLM is a classic symbolic system that generates symbols according to rules,
but instead it's a system that generates symbols according to recorded probabilities. The focus here is on symbols: the LLM reasoning starts with <think> token, ends with </think> token, and everything in between is symbols (tokens). Quite different from how human brain works, where symbols are not that much involved in reasoning.
> The article asserts general guarantees of program correctness have fundamental limits, but it explicitly says those limits apply to humans too.
Yep, however I would like to assert that today we have evidence of systems architected by humans and behaving correctly (like Linux kernel), while there are no evidence of systems architected by LLMs that behave correctly. Even the systems which are architected by LLMs with human assistance (such as Claude Code or Codex) routinely have horrendous bugs (you're welcome to search "Claude Code Bug" and "Codex Bug" on this very website).
> It certainly does not present a fundamental problem with LLMs as they exist today.
The article was created 10 years ago that is why it does not address LLMs. However it asserts that mathematics modeling is not that helpful for writing correct programs. As long as LLM reasoning is closer to mathematics modeling than human mental modeling, LLMs will be worse at writing correct programs than humans. Some future technology less focused on symbolic reasoning could change that, but today it's not on sufficient level.
Yeah, good question, thanks. Llm reasoning is done exclusively in the way of symbolic manipulation. Human reasoning is done not in symbols but in mental models that are hard to explain for today science but we know they are somewhat efficient for producing correct programs. Ron's post makes the case to explain that symbolic reasoning is not sufficient for producing correct programs. Hope this helps.
It is combinatorial explosion of possible states (not as in memory state of program, but the differing visions of the next state of architecture). Plus high impact of mistakes, suboptimal solutions and randomness when switching between states.
Edit: I like this 10 years old blog post in regards to explaining the problem much better than I could do: https://pron.github.io/posts/correctness-and-complexity
Its not happening, at least not with LLMs, maybe some future technology will be better with that, but not language models. Writing such a complex software as database engine requires strategy, awareness and rigor. It is somewhat similar to winning a sweaty game of chess with an experienced opponent. LLMs are great with broadness of their knowledge, matching patterns and synthesising that knowledge, but they fail with strategy and awareness part. And this is why they are bad in chess.
CPU and memory are not the only concerns,
file system and network access are dangerous as well,
and they tend to be less instrumented in terms of hardware isolation.
I do it like this, but on one computer, using QEMU. I have 3 different VMs for
- development & AI tools
- infra rollouts
- banking / other sensitive services
I have Steam Deck for gaming, and do most of casual web browsing still on host, because it's secure enough to use Firefox with ublock + privacy badger and enforcing SELinux. However, I dont use socials other than HN and Reddit under pseudonymous accounts. If I had real zuckerbook or instacrap accounts, maybe I'd have another VM for socials too.
The issue could be resolved by introducing some sort of signing for openvsx extension packages. Then regardless of from where one downloaded Claude, it can be always verified to be authentic or not. This technology exists for ages and prominently featured in all mainstream Linux distributions. Cursor, Anthropic and other openvsx users need to work with Eclipse Foundation to make this happen.
Boris: "By winter 2025, 100 percent of my code was written by Claude Code. Not a single line by hand."
Igor Kofman: "I do not write any code by myself anymore."
Turns out if you are not writing the code, then North Koreans might do it for you
Wow, great notion about using git remote to transfer source from the agent host. I already use ssh into VM to transfer the files like designs for sharing, so going to adopt your approach right now :)
Well for me, as developer on a modern stack, there is always some risk involved. If not from agentic tools, then the package registries are compromised left and right, so it all comes down to mitigating the risk. I use Claude, Codex and Cursor/SpaceX agents, and Cursor comes very convenient to have all three in a single interface. Also I still code manually especially when prototyping, so having an IDE helps with it as well. I knew about OpenVSX situation with the search, so when installing extensions I always did triple-checking to verify the extension being really submitted by upstream and not by squatter; but now given upgrades can also deliver a compromise I am thinking of developing an extension that will act in similar manner to pnpm minimum release age.
I only use agentic development tools in qemu VM that has no production
access or host shared folders, and the only secrets there are development.
I still have my Git ssh key there, but I never use ssh agent to load the keys.
The worst that agentic tools can do to me, is to install keylogger and compromise
my ssh key and password, but even then my deployment workflow includes creating release in Github,
which is not possible via ssh key only. I was very lucky to avoid the hack yesterday, as I use Claude installed via OpenVSX, but even if I installed the compromised version, the damage would be minimal.
There is no alternative really. Cursor was the market leader in LLM-assisted IDE experience before SpaceX acquisition, though Elon nerfed the product a lot in recent months. And there is still no
good alternative to it. Google tries to compete via Antigravity, but that also uses OpenVSX. I myself use Cursor and use Claude there installed via OpenVSX, and was not affected only thanks to my laziness (I saw upgrade button, but became distracted and did not click). I hope this situation will become a trigger to improve OpenVSX security.
Yeah OpenVSX security is a joke, they also have dumb search algorithm which
allows for easy squatting of similar-looking extension names so they show up in search first when you look for official extensions. There were quite a few prominent hacks done that way. I think Microsoft likes OpenVSX situation because it allows them to claim
their proprietary VSCode marketplace to be more secure.
Cursor always used OpenVSX for VSCode extensions. There is separate "Cursor Marketplace" (
https://cursor.com/marketplace) but it installs "Cursor Plugins" which are exclusively Cursor feature not related to VSCode, and Anthropic had never released a "Cursor Plugin"
Nah it was Anthropic, see my comment at
https://news.ycombinator.com/reply?id=50302513&goto=item%3Fi...Cursor does a bad reporting branding OpenVSX as "Cursor Marketplace". What
a joke the company had become under Elon. But also it might be related to
Anthropic/SpaceX business ties, they try to minimize damage on Anthropic
pre-IPO
Cursor uses OpenVSX marketplace (
https://open-vsx.org/), that all non-Microsoft forks of VSCode utilize
(official marketplace is only allowed in official proprietary VSCode release). It's not really related to Cursor/SpaceX in any way.
I suspect it's still Anthropic who were hacked, because they control OpenVSX
account with release privileges.
edit:
Binance confirms my suspicions, via https://www.binance.com/en/square/post/10-30-2026-claude-cod...
The affected Open VSX extension is owned by Anthropic employee Igor Kofman, but it is unclear whether his account or another part of the publishing process was compromised.
The point of cold wallet is to minimize movements, if there is high frequency of funds moved from a wallet then it is likely can not be called cold any more. Plus CEX in any case still needs to have large buffer of funds on some wallet to support day trading.
The hack is publicized for at least a hour, but from reporting I saw they still continue to drain funds as for now. I wonder why large CEX do not have any safety measures for this kind of hacks, they could withdraw everything to newly generated wallets and minimize damage that way. It is certainly not the first time CEX wallets get hacked.
Ukraine uses Starshield, which is paid by Germany (with current contract ending in April 2027). What you describe in regards to chip hacking so it reports location as static, it is how
nazi invaders got it working in a small chunk of time during 2025, before whitelisting was
introduced for the terminals.
I wonder if Web, WebPKI and some other global network infrastructure are a sign of the emerging internationalist, stateless order ahead of us. Might be a hot topic on next Joe Rogan show :)
Just saw your comment, thanks, not sure if putins regime can be met with anything but hatred, and not only because of its treatment of Ukrainians, but also many Russians are as well victims of nazi kgb regime in direct or indirect way. For many people born in Russia, overthrow of that regime might be the only way to achieve better future for their family and their native region. Ukraine can achieve peace, forget about Russia existence for a while, and have minimum hatred for russians going forward, like Finland did, but for those people who were unlucky to be born in Russia, destroying kgb nazi is the only possible way forward, apart from the alternative of life in exile (like many did in 20th century during the original kgb regime). Same goes for Belarus as well. The most "Russophobe" people (in terms of kgb propaganda ofc) I've ever met were Russians born in Russia.
Previously such incidents have lead to the CA business cessation by the involved entities (e.g. Diginotar case). I dont see how it would be different in this case.
Yeah, I have extensive collection of prompts for reviews and usually run 3-5 (automated) review takes on large sets of changes before doing eyeball reviews. Auto reviews indeed find many obvious bugs, but they usually slip through things that are less
obviously stupid but are bad conceptually, for example, using persistence / queue / repository in a place where set of pure functions would be enough; or vice-versa, not persisting the state that must be persisted for reasons, etc.
Another problem is that review changes can easily introduce new issues, so my review prompts go like that: if you find outright shit, fix it right away, but if you have slightest concern, just write your ideas and suggestions in the review and let me decide then. I try from time to time to omit the "slightest concern" part and make agent do all the shit it wants, and usually end up reverting those changes.
But as the sibling comment said, not all code needs quality and indeed I skip some code too, like user
playwright journeys -- I only need one auto review pass to verify they indeed test things according to a doc, and I'm fine not reading them. Not all code like that though, some stuff would wreck havoc later if its not looked at.
> Whereas if you're handing off all dev and no longer reading code, just making CTO-level architecture decisions, then there are only 2~3 models right now worth using, none cheap.
I use frontier models every day and night, but not sure if the described approach is possible with any of them right now. Models still make stupid mistakes and people who do not read the code produced are bound for surprising discoveries in their codebase. Not trying to downplay the effectiveness of LLM-assisted coding and all that, but it's hard to imagine how much stupid shit i would've committed if I was not reading their output.
The right only cares about wellbeing of the rich, and is usually keen to exploit the demographic majority in the interest of the few (see what is happening in US now). I follow politics of democratic countries closely for few dozen years, but I have yet to find an example of rightist party which improved economic wellbeing of its constituents (except those who are already rich)
Interesting! I am from Donetsk myself, but left in 2014. I am pretty much united with my Russian friends on the topic of destroying putin's regime. Not sure if any other unity makes sense now.