News for hackers. Stuff that matters.
Re:Here’s how you can build and train GPT-2 from scratch usi... by elpocko on Saturday July 06, 2024 @05:47AM (#40889308)
It's gotta be an exceptionally awesome video to beat written text though. I'll take mediocre text over good video any day.
Re:Here’s how you can build and train GPT-2 from scratch usi... by throwaway71271 on Saturday July 06, 2024 @06:58AM (#40889610)
This particular video is an exceptionally awesome video.

And so are his other neural net videos.

He keeps bringing you back to first principles, again and again, does not let a single doubt remain, and the code is very accessible and practical.

The intersection between those who can teach and those who truly understand the deepest essence of a subject is much smaller than we would hope. People like Feynman, Tanenbaum, Sussman, Susskind, and now Karpathy are exceedingly rare. Each of them is a gift for generations to come. So, when you find one whose style resonates with the way you think, I suggest watching their videos multiple times. :)

Re:Here’s how you can build and train GPT-2 from scratch usi... by frognumber on Saturday July 06, 2024 @06:50AM (#40889574)
Convert video to text with this one simply secret trick:

* yt-dlp

* whisper

(whisper is surprisingly good for a lot of educational videos)

Even better is to connect text to images. but that's less of a simple trick (although I do have the code).

Re:Here’s how you can build and train GPT-2 from scratch usi... by omerhac on Saturday July 06, 2024 @07:18AM (#40889708)
Thats cool but not exactly the same as reading text that was written to be consumed as text. I think a big part of the reason technical text is easier to digest is the way it's written, and not so much the medium.
Re:Here’s how you can build and train GPT-2 from scratch usi... by nottorp on Saturday July 06, 2024 @06:05AM (#40889376)
How can a video be better than text based info when we're talking about a programming task, which is all text?
Re:Here’s how you can build and train GPT-2 from scratch usi... by hombre_fatal on Saturday July 06, 2024 @06:45AM (#40889555)
Because the value isn't in just having raw reference material like complex code you just paste into your program. The value is in communicating ideas and understanding.

You don't need to skim through the 4 hour video long to see it's full of whys and hows and explanations and demonstrations that massively dwarf the blog post. It's basically a mini course.

Programming is an interactive process, so transcribing the video to text also removes a lot information. The video isn't just an audio form of what could've been a long text tutorial. You're watching someone do something.

Re:Here’s how you can build and train GPT-2 from scratch usi... by nottorp on Saturday July 06, 2024 @07:05AM (#40889651)
> The value is in communicating ideas and understanding.

And those ideas absolutely can't be communicated in concise writing, I have to waste my life on listening to talking heads?

Re:Here’s how you can build and train GPT-2 from scratch usi... by hombre_fatal on Saturday July 06, 2024 @07:10AM (#40889673)
Yes. Skim the video. You're watching someone explain things interactively in a way that cannot be done with text.

Whether you have the preference for it or not is not up for debate here.

Re:Here’s how you can build and train GPT-2 from scratch usi... by nottorp on Saturday July 06, 2024 @07:30AM (#40889766)
> Skim the video.

So not all 4 hours are worth watching? :)

Re:Here’s how you can build and train GPT-2 from scratch usi... by jamescmartinez on Saturday July 06, 2024 @10:02AM (#40890537)
This is an option for those who want to learn from a more interactive medium instead of from a textbook.

Different people learn in different ways. I wouldn’t call any of it a waste.

Re:Here’s how you can build and train GPT-2 from scratch usi... by graovic on Saturday July 06, 2024 @06:08AM (#40889399)
You can listen someone thought process, that is extremely valuable