News for hackers. Stuff that matters.
Re:AnyGPT: Unified Multimodal LLM with Discrete Sequence Mod... by cs702 on Wednesday February 21, 2024 @10:49AM (#39455269)
The key takeaway for me:

Whether data is continuous or discrete, no matter its modality (text, video, music, etc.), we now have an array of proven methods for representing it with discrete tokens, enabling us to use existing sequence modeling architectures (Transformers, linear RNNs).

We live in interesting times!

Re:AnyGPT: Unified Multimodal LLM with Discrete Sequence Mod... by aaron_wang on Thursday February 22, 2024 @05:06AM (#39465191)
Totally agree. Another takeaway is that data from different modality has some shared intrinisity, so one can be mapped to another.