News for hackers. Stuff that matters.
Re:Pathways Language Model (PaLM): Scaling to 540B parameters by oofbey on Monday April 04, 2022 @06:27PM (#30912801)
Why is that tragic? Classic NLP tasks are IMHO kinda pointless. Nobody _actually_ cares about parse trees, etc. These things were useful when that was the best we could do with ML, because they allowed us to accomplish genuinely-useful NLP tasks by writing code that uses things like parse trees, NER, etc. But why bother with parse trees and junk like that if you can just get the model to answer the question you actually care about?
Re:Pathways Language Model (PaLM): Scaling to 540B parameters by hashingroll on Wednesday April 06, 2022 @01:17AM (#30928736)
I would not discount NLP tasks just yet. In practice, they are still used to solve problems like spellcheck, autocomplete and even words and text rewrites.
Re:Pathways Language Model (PaLM): Scaling to 540B parameters by oofbey on Wednesday April 06, 2022 @05:14PM (#30937492)
I'd argue that every one of those tasks is better solved with a neural LM. The fact that people still use traditional techniques does not mean those old techniques are better. It just means those tasks haven't caught up to the modern era yet.

Or maybe it's because MLM are too expensive to run. But that's hardly an argument that MLM should solve traditional NLP tasks.

Re:Pathways Language Model (PaLM): Scaling to 540B parameters by The_rationalist on Tuesday April 05, 2022 @11:37AM (#30920357)
if you can just get the model to answer the question you actually care about? LOL your level of optimism is over 9000. Language models are shit, they regularly utter nonsense and have an impressive number of limitations (e.g. no continual learning). A neuro-symbolic system on the other hand can be incrementally improved upon, has continual learning/a memory, and is interpretable.
Re:Pathways Language Model (PaLM): Scaling to 540B parameters by oofbey on Wednesday April 06, 2022 @05:13PM (#30937475)
You selling buggy whips? Good luck with that.