News for hackers. Stuff that matters.
Re:Transformers for software engineers by chaosprint on Saturday April 02, 2022 @04:37PM (#30891470)
Very good content! It's interesting that you use Rust here.
Re:Transformers for software engineers by nelhage on Saturday April 02, 2022 @06:32PM (#30892265)
(author here)

I actually did a v0 writeup in Go, but I wanted a language that had a bit more powerful type system and more support for a fluent/functional style in some of the expressions. I was optimizing for what I know and felt was highly expressive; I've since gotten a bunch of feedback from people who are interested in the content but not comfortable reading Rust, so perhaps it wasn't the best choice.

Re:Transformers for software engineers by ShamelessC on Sunday April 03, 2022 @02:33AM (#30894931)
What does the`.0` calling convention mean in Rust? e.g. ``` for (i, r) in right.0.iter().enumerate() { out.0[i] += r; } ```

Aside from that, I thought it was fairly legible. Great write-up by the way. Squashing things into state helps get rid of some of the spookiness created by matrix multiplication and back-propagation. I also really appreciated seeing the explanation on the actual MLP part of the transformer as that is typically assumed to be prior knowledge in other tutorials.

Re:Transformers for software engineers by heinrichf on Sunday April 03, 2022 @02:47AM (#30894983)
It's to access the tuple's single element, but the author could have used #[repr(transparent)]
Re:Transformers for software engineers by mrfusion on Saturday April 02, 2022 @08:14PM (#30892882)
You should mention that you’re using rust. I got distracted trying I figure out what language you were using.
Re:Transformers for software engineers by mrfusion on Saturday April 02, 2022 @08:13PM (#30892876)
Python would be awesome!
Re:Transformers for software engineers by f311a on Saturday April 02, 2022 @05:30PM (#30891856)
I think it’s better to use other languages for pseudo code. Rust is not the easiest language to read and understand for people who never used it.