News for hackers. Stuff that matters.

A comment on Ask HN: If we train an LLM with “data” instead of “language” tokens. Parent comment

Re:Ask HN: If we train an LLM with “data” instead of “langua... by arbuge on Wednesday August 16, 2023 @01:17PM (#37150275)
It doesn't seem to me that that's a problem. You just put data in and expect data out.

For example, a weather continuation model could take in all the available weather data up to the present moment and provide a continuation of that, i.e. a weather forecast.

Likewise financial market predictions etc.

[ Parent | on HN ]
Re:Ask HN: If we train an LLM with “data” instead of “langua... by BreadPants on Wednesday August 16, 2023 @01:30PM (#37150516)
How about a multi-headed approach. LDMs specializing in certain data operating under one LLM that acts as our point of access?
[ Parent | on HN ]
Re:Ask HN: If we train an LLM with “data” instead of “langua... by MichaelZuo on Wednesday August 16, 2023 @01:24PM (#37150414)
That's a good point, Math is just as much a communicable language as English, French, Japanese, etc... maybe even more so according to some formal definitions.
[ Parent | on HN ]
Re:Ask HN: If we train an LLM with “data” instead of “langua... by T-A on Wednesday August 16, 2023 @02:08PM (#37151168)
> a weather continuation model could take in all the available weather data up to the present moment and provide a continuation of that

And it turns out to work really well:

https://arxiv.org/abs/2212.12794

predicts hundreds of weather variables, over 10 days at 0.25 degree resolution globally, in under one minute

[ Parent | on HN ]