of course. you know how i know? absolutely no one except the wannabees has time to read papers - people working in the area have deadlines and meetings. we absorb the content of the paper by osmosis - convos, code bases, occasionally a talk at a conference.
it's especially horrible advice from the perspective of pedagogy to tell a n00b to read a paper (so the person giving the advice has immediately disqualified themselves from possibly being an expert) because papers are horribly written, omit critical details, and function purely as advertising for the authors, group, etc.
for every poor undergrad/n00b soul reading this comment, take this thing to heart that took me too long to unlearn (due to its constant perpetuation by people like gop): reading the paper is 100:1 waste-of-time:value-derived.
if i hear about/see some paper that makes strong claims that are relevant to my work, i will look for a github link and/or email the first author. 5/10 i get a response (ratio is going up as i'm getting to be more ingratiated in my community). the other times i just move on - none of these papers have some revolutionary cure for cancer in them so most of the time what i'm already doing is already close enough that i don't need to kill myself figuring out the new thing.
that paper in particular (attention) has nothing in it that is in the least bit interesting/revolutionary. the hard part of attention isn't writing down softmax(QK^T)V, the hard part is executing that matrix product fast enough that you're not waiting eons for your model to converge.
Everytime I do a foundational activity like this it does turn out to be illuminating. Why do you doubt you'll learn something?
Also will add the models turned out to be a lot simpler to understand than I expected going in.
(I totally realize this is subjective, but that has been my experience with my own learning in the space over the last few years as well as some folks I've mentored)
Is the person writing it just repeating something they read? Is it just because they like the ´coding from first principles’ aesthetics?
I mean let’s imagine that someone does read that paper, and manage to replicate the code (quite an effort from someone coming from outside AI and academia).
Then what? I doubt it’s particularly illuminating. That doesn’t really qualify for a job by itself. So what’s the goal there? Is it just a thing to say to look like a cool hacker that code from scratch?