News for hackers. Stuff that matters.
Re:RWKV RNN: Better than ChatGPT? by RobotToaster on Thursday March 23, 2023 @07:41PM (#35283107)
I wonder if we will see LLMs on FPGAs or FPGA like devices.

Crypto went from graphics cards to ASICs, we may see something similar with LLMs given the hype.

Re:RWKV RNN: Better than ChatGPT? by pffft8888 on Thursday March 23, 2023 @08:01PM (#35283268)
I always think of FGPAs for things you want to update without having to repurchase the hardware. If the FPGA inside a gadget can be reprogrammed over the Internet then I think it's more suited than an ASIC for LLMs.
Re:RWKV RNN: Better than ChatGPT? by int_19h on Friday March 24, 2023 @06:54PM (#35296775)
It could go even further, in theory. The kind of ops that the current crop of LLMs needs is very simple, and at the same time there's no hard requirement for precision (which is why 4-bit quantization works so well). This means that unconventional approaches such as analog computing are potentially in the play again - it's easy to do addition and multiplication in an analog circuit, if you don't care about the answer being precise, and in theory one could pack a lot more of those circuits in the same space.
Re:RWKV RNN: Better than ChatGPT? by smaddox on Saturday March 25, 2023 @03:58PM (#35306625)
There's already at least one company with a product: Mythic AMP https://mythic.ai/products/m1076-analog-matrix-processor/

And there's several research papers about other architectures. For example: https://arxiv.org/abs/2205.05853