News for hackers. Stuff that matters.
Re:Alice in Wonderland: Simple Tasks Showing Complete Reason... by skydhash on Wednesday June 05, 2024 @03:00PM (#40588925)
The best compression relies on understanding. What LLM is is mostly data how humans use words. We understand how to make this data (which is a compression of human text) and use it (generate something). AKA it’s “production rules”, but statistical.

The only issue is ambiguity. What can be generated strongly depends on the order of the tokens. A slight variation can change the meaning and the result is worthless. Understanding is the guardrail against meaningless statement and LLMs lack it.

Re:Alice in Wonderland: Simple Tasks Showing Complete Reason... by tsunamifury on Wednesday June 05, 2024 @03:08PM (#40589025)
You seem to entirely miss how attention layers work...