Re:AI Predictions for 2025, from Gary Marcus
by mordymoop on Thursday January 02, 2025 @10:39AM
(#42575339)
I agree with what you say here, but I still don’t feel like this gives Marcus the edge here, because Perplexity does a fantastic job at finding correct information and, in my experience, never hallucinates. I would call Perplexity and whatever they’re doing a pretty “robust” solution to hallucination.
So there are solutions to hallucination, but the product has to prioritize that to the extent that it reduces the model’s effectiveness as a general tool.
ChatGPT/Claude still sucks at providing factual info. You can see how in some areas (like coding), massive amounts of training has taken place, and the model is almost always right when recalling existing knowledge, particularly if you ask about the major frameworks. Then there are somewhat more obscure topics, where the models are mostly right, but can be convincigly wrong, and it's very hard to tell which is which.
If you ask about something that wasn't deemed to have economic value, such as book recommendations (I thought a model that was advertised to have been trained on the entirety of human literaly works would be good at that), what you get is almost always obviously wrong info.