maxbbraun
Re:Show HN: Llama Running on a Microcontroller
on Wednesday November 15, 2023 @05:13PM (#38283331)
Just to clarify: Inference is happening on the Arm Cortex-M7. The Coral TPU chip is off in this implementation.
Re:Show HN: Llama Running on a Microcontroller
on Wednesday November 15, 2023 @12:27PM (#38279497)
I was wondering if it's possible to fit a non-trivial language model on a microcontroller. Turns out the answer is some version of yes!
This project is using the Coral Dev Board Micro with its FreeRTOS toolchain. The board has a number of neat hardware features not currently being used here (notably a TPU, sensors, and a second CPU core). It does, however, also have 64MB of RAM. That's tiny for LLMs, which are typically measured in the GBs, but comparatively huge for a microcontroller.
The LLM implementation itself is an adaptation of llama2.c and the tinyllamas checkpoints trained on the TinyStories dataset. The quality of the smaller model versions isn't ideal, but good enough to generate somewhat coherent (and occasionally weird) stories.
Re:32“ E Ink screen that displays daily newspapers on your wall
on Friday July 07, 2023 @09:13AM (#36631243)
Search Wittgenstein's Tractatus with GPT-3 embeddings
Re:E-ink art frames are a scam (2021)
on Sunday December 04, 2022 @04:59PM (#33858406)
Photos are tricky, but you can make e-paper displays look nice if you embrace the constraints. For example: https://accent.ink
Philosophy is the theory of the nonsense of life
Wittgenstein 2022
Inside X’s Mission to Make Robots Boring
Let's Make More Calm Technology
Smarter Mirrors and How They’re Made
This Machine Turns Trump Tweets into Planned Parenthood Donations
Re:My Bathroom Mirror Is Smarter Than Yours
on Monday February 01, 2016 @12:28PM (#11013319)
That's exactly what I focused on. :) I picked a very thin display panel and then hid it between the door and the glass. I'll probably post some more behind-the-scenes pictures of that.