News for hackers. Stuff that matters.

A comment on Ask HN: If we train an LLM with “data” instead of “language” tokens.

Re:Ask HN: If we train an LLM with “data” instead of “langua... by jmbiven on Wednesday August 16, 2023 @01:02PM (#37150033)
Jordan Volz wrote an article speculating about this earlier this year. First time I heard the term Large Data Model, which I love and plan on using.

https://medium.com/@jordan_volz/who-owns-the-future-looking-...

"I won’t claim it’s simple, but as we’ve built AI that can understand human language, we can similarly build AI that understands data...Instead of building a large language model, we instead can build large data models (LDMs?)."

[ Parent | on HN ]
Re:Ask HN: If we train an LLM with “data” instead of “langua... by croes on Wednesday August 16, 2023 @01:11PM (#37150176)
But how do we communicate with an entity that understands data but not human language?
[ Parent | on HN ]
Re:Ask HN: If we train an LLM with “data” instead of “langua... by arbuge on Wednesday August 16, 2023 @01:17PM (#37150275)
It doesn't seem to me that that's a problem. You just put data in and expect data out.

For example, a weather continuation model could take in all the available weather data up to the present moment and provide a continuation of that, i.e. a weather forecast.

Likewise financial market predictions etc.

[ Parent | on HN ]
Re:Ask HN: If we train an LLM with “data” instead of “langua... by BreadPants on Wednesday August 16, 2023 @01:30PM (#37150516)
How about a multi-headed approach. LDMs specializing in certain data operating under one LLM that acts as our point of access?
[ Parent | on HN ]
Re:Ask HN: If we train an LLM with “data” instead of “langua... by MichaelZuo on Wednesday August 16, 2023 @01:24PM (#37150414)
That's a good point, Math is just as much a communicable language as English, French, Japanese, etc... maybe even more so according to some formal definitions.
[ Parent | on HN ]
Re:Ask HN: If we train an LLM with “data” instead of “langua... by T-A on Wednesday August 16, 2023 @02:08PM (#37151168)
> a weather continuation model could take in all the available weather data up to the present moment and provide a continuation of that

And it turns out to work really well:

https://arxiv.org/abs/2212.12794

predicts hundreds of weather variables, over 10 days at 0.25 degree resolution globally, in under one minute

[ Parent | on HN ]
Re:Ask HN: If we train an LLM with “data” instead of “langua... by optimalsolver on Wednesday August 16, 2023 @01:29PM (#37150500)
Send it a sequence of raw UTF bytes and see what it replies with.
[ Parent | on HN ]