News for hackers. Stuff that matters.
Re:Cursor IDE Support Hallucinates Lockout Policy Causes Mas... by anonzzzies on Tuesday April 15, 2025 @11:13PM (#43701061)
Did anyone say that? They are an issue everywhere, including for code. But with code at least I can have tooling to automatically check and feed back that it hallucinated libraries, functions etc, but with just normal research / problems there is no such thing and you will spend a lot of time verifying everything.
Re:Cursor IDE Support Hallucinates Lockout Policy Causes Mas... by manmal on Wednesday April 16, 2025 @12:32AM (#43701495)
You get some superficial checking by the compiler and test cases, but hallucinations that pass both are still an issue.
Re:Cursor IDE Support Hallucinates Lockout Policy Causes Mas... by anonzzzies on Wednesday April 16, 2025 @12:37AM (#43701523)
Absolutely, but at least you have some lines of defence while with real world info you have nothing. And the most offending stuff like importing a package that doesn't exist or using a function that doesn't exist does get caught and can be auto fixed.
Re:Cursor IDE Support Hallucinates Lockout Policy Causes Mas... by skyfaller on Wednesday April 16, 2025 @02:09AM (#43702019)
Such errors can be caught and auto-fixed for now, because LLMs haven't yet rotted the code that catches and auto-fixes errors. If slop makes it into your compiler etc., I wouldn't count on that being true in the future.
Re:Cursor IDE Support Hallucinates Lockout Policy Causes Mas... by threeseed on Wednesday April 16, 2025 @12:57AM (#43701637)
I use Scala which has arguably the best compiler/type system with Cursor.

There is no world in which a compiler or tooling will save you from the absolute mayhem it can do. I’ve had it routinely try to re-implement third party libraries, modify code unrelated to what it was asked, quietly override functions etc.

It’s like a developer who is on LSD.

Re:Cursor IDE Support Hallucinates Lockout Policy Causes Mas... by Tainnor on Wednesday April 16, 2025 @04:17AM (#43702854)
I don't know Scala. I asked cursor to create a tutorial for me to learn Scala. It created two files for me, Basic.scala and Advanced.scala. The second one didn't compile and no matter how often I tried to paste the error logs into the chat, it couldn't fix the actual error and just made up something different.
Re:Cursor IDE Support Hallucinates Lockout Policy Causes Mas... by Terr_ on Wednesday April 16, 2025 @04:07AM (#43702783)
Yeah, everyone wanted a thinking machine, but the best we can do right now is a dreaming machine... And dreams don't have to make sense.
Re:Cursor IDE Support Hallucinates Lockout Policy Causes Mas... by jmaker on Wednesday April 16, 2025 @04:27AM (#43702917)
Granted the Scala language is much more complex than Go. To produce something useful it must be capable of an equivalent of parsing the AST.
Re:Cursor IDE Support Hallucinates Lockout Policy Causes Mas... by larodi on Wednesday April 16, 2025 @05:48AM (#43703401)
Developer on LSD is likely to hallucinate less in terms of how weird the LLM hallucinations are sometimes. Besides I know people, not myself, who fare very well on LSD and particularly when micro dosing Adderal style
Re:Cursor IDE Support Hallucinates Lockout Policy Causes Mas... by miningape on Wednesday April 16, 2025 @07:22AM (#43703966)
Mushrooms too! I find they get me into a flow state much better than acid (when microdosing).
Re:Cursor IDE Support Hallucinates Lockout Policy Causes Mas... by felipefar on Wednesday April 16, 2025 @01:02AM (#43701674)
Yes, most people who have an incentive in pushing AI say that hallucinations aren't a problem, since humans aren't correct all the time.

But in reality hallucinations either make people using AI lose a lot of their time trying to stuck the LLMs from dead ends or render those tools unusable.

Re:Cursor IDE Support Hallucinates Lockout Policy Causes Mas... by jimbokun on Wednesday April 16, 2025 @10:00AM (#43705676)
> Yes, most people who have an incentive in pushing AI say that hallucinations aren't a problem, since humans aren't correct all the time.

We have legal and social mechanisms in place for the way humans are incorrect. LLMs are incorrect in new ways that our legal and social systems are less prepared to handle.

If a support human lies about a change to policy, the human is fired and management communicates about the rogue actor, the unchanged policy, and how the issue has been handled.

How do you address an AI doing the same thing without removing the AI from your support system?

Re:Cursor IDE Support Hallucinates Lockout Policy Causes Mas... by anonzzzies on Thursday April 17, 2025 @03:05AM (#43713912)
Still fine the company who uses the AI? It cannot be prevented with the current state of AIs so you will need a disclaimer and, if a user cancels, show the latest support chats in the crm for that user so you can add a human in the mix.
Re:Cursor IDE Support Hallucinates Lockout Policy Causes Mas... by Gormo on Wednesday April 16, 2025 @11:25AM (#43706656)
> Yes, most people who have an incentive in pushing AI say that hallucinations aren't a problem, since humans aren't correct all the time.

Humans often make factual errors, but there's a difference between having a process to validate claims against external reality, and occasionally getting it wrong, and having no such process, with all output being the product of internal statistical inference.

The LLM is engaging in the same process in all cases. We're only calling it a "hallucination" when its output isn't consistent with our external expectations, but if we regard "hallucination" as referring to any situation where the output for a wholly endogenous process is mistaken for externally validated information, then LLMs are only ever hallucinating, and are just designed in such a way that what they hallucinate has a greater than chance likelihood of representing some external reality.

Re:Cursor IDE Support Hallucinates Lockout Policy Causes Mas... by rini17 on Wednesday April 16, 2025 @04:15AM (#43702841)
Except when the hallucinated library exists and it's malicious. This is actually happening. Without AI, by using plain google you are less likely to fall for that (so far).
Re:Cursor IDE Support Hallucinates Lockout Policy Causes Mas... by jmaker on Wednesday April 16, 2025 @04:25AM (#43702901)
Until the model injects a subtle change to your logic that does type-check and then goes haywire in production. Just takes a colleague of yours under pressure and another one to review the PR, and then you’re on call and they out sick or on vacation.