News for hackers. Stuff that matters.
Re:Building LLMs from the Ground Up: A 3-Hour Coding Workshop by menzoic on Sunday September 01, 2024 @04:00AM (#41415117)
Is this a joke? Can’t tell. OpenAI uses PyTorch to build LLMs
Re:Building LLMs from the Ground Up: A 3-Hour Coding Workshop by jnhl on Sunday September 01, 2024 @04:16AM (#41415173)
You could always go deeper and from some points of view, it's not "from the ground up" enough unless you build your own autograd and tensors from plain numpy arrays.
Re:Building LLMs from the Ground Up: A 3-Hour Coding Workshop by 0cf8612b2e1e on Sunday September 01, 2024 @12:47PM (#41418309)
Numpy sounds like cheating on the backs of others. Going to need your own hand crafted linear algebra routines.
Re:Building LLMs from the Ground Up: A 3-Hour Coding Workshop by TZubiri on Sunday September 01, 2024 @04:29AM (#41415232)
Source please?
Re:Building LLMs from the Ground Up: A 3-Hour Coding Workshop by leobg on Sunday September 01, 2024 @05:04AM (#41415371)
People think of the Karpathy tutorials which do indeed build LLMs from the ground up, starting with Python dictionaries.
Re:Building LLMs from the Ground Up: A 3-Hour Coding Workshop by krmboya on Sunday September 01, 2024 @09:21AM (#41416801)
From scratch is relative. To a python programmer, from scratch may mean starting with dictionaries but a non-programmer will have to learn what python dicts are first.

To someone who already knows excel, from scratch with excel sheets instead of python may work with them.

Re:Building LLMs from the Ground Up: A 3-Hour Coding Workshop by wredue on Sunday September 01, 2024 @12:26PM (#41418144)
For the record, if you do not know what a dict actually is, and how it works, it is impossible to use it effectively.

Although if your claim is then that most programmers do not care about being effective, that I would tend to agree with given the 64 gigs of ram my basic text editors need these days.

Re:Building LLMs from the Ground Up: A 3-Hour Coding Workshop by carlmr on Sunday September 01, 2024 @04:32PM (#41420129)
>For the record, if you do not know what a dict actually is, and how it works, it is impossible to use it effectively.

While I agree it's good to know how your collections work. "Efficient key-value store" may be enough to use it effectively 80% of the time for somebody dabbling in Python.

Sadly I've met enough people that call themselves programmers that didn't even have such a surface level understanding of it.

Re:Building LLMs from the Ground Up: A 3-Hour Coding Workshop by atoav on Sunday September 01, 2024 @05:45AM (#41415547)
No it is not. From scratch has a meaning. To me it means: in a way that letxs you undrrstand the important details, e.g. using a programming language without major dependencies.

Calling that from scratch is like saying "Just go to the store and tell them what you want" in a series called: "How to make sausage from scratch".

When I want to know how to do X from scratch I am not interested in "how to get X the fastest way possible", to be frank I am not even interested in "How to get X in the way others typically get it", what I am interested in is learning how to do all the stuff that is normally hidden away in dependencies or frameworks myself — or, you know, from scratch. And considering the comments here I am not alone in that reading.

Re:Building LLMs from the Ground Up: A 3-Hour Coding Workshop by kenjackson on Sunday September 01, 2024 @12:49PM (#41418320)
Your definition doesn’t match mine. My definition is fuzzier. It is “building something using no more than the common tools of the trade”. The term “common” is very era dependent.

For example, building a web server from scratch - I’d probably assume the presence of a sockets library or at the very least networking card driver support. For logging and configuration I’d assume standard I/o support.

It probably comes down to what you think makes LLMs interesting as programs.

Re:Building LLMs from the Ground Up: A 3-Hour Coding Workshop by atoav on Monday September 02, 2024 @07:15AM (#41424404)
It is okay to differ on this. Language is not an exact science. It is however always good to factor in expectations when you describe things.

E.g. when a title says it shows you how to do a thing in vanilla javascript from scratch bringing in jquery in the first step makes that tile a lie. If you bring in a hefty dependency on step 1 and run three imported function the vanilla javascript part might be fine, but the from scratch starts to do some heavy lifting.