News for hackers. Stuff that matters.
Re:NY Times copyright suit wants OpenAI to delete all GPT in... by vanviegen on Thursday December 28, 2023 @04:42AM (#38791729)
> Gradient descent and backpropagation don't take place in the brain.

Not exactly, no, but the 'neurons that fire together wire together' way of learning has a pretty similar effect.

> LLMs "learn" in the same way that Excel sheets "learn".

I've never seen an excel sheet do anything like backpropagation.

Re:NY Times copyright suit wants OpenAI to delete all GPT in... by zimpenfish on Thursday December 28, 2023 @05:34AM (#38792053)
> I've never seen an excel sheet do anything like backpropagation.

Not strictly in the sense you mentioned (assuming that you mean "by themselves") but people may find [1] and [2] interesting.

[1] https://pub.towardsai.net/building-a-neural-network-with-bac...

[2] https://towardsdatascience.com/demystifying-feed-forward-and...

Re:NY Times copyright suit wants OpenAI to delete all GPT in... by galangalalgol on Thursday December 28, 2023 @12:21PM (#38795763)
Sadly, I have seen one. It was a vba script from the late 90s that used a simple dense multilayer network to do some unsupervised pattern classification. The linear algebra tools in vba/excel along with the solvers are all native dll code and the vba itself is all AOT compiled to native, so it typically runs very fast, and for small matrices it beats out numpy by an order of magnitude due to the ffi overhead. Was it the wrong tool? It depends on your constraints, but probably. It did work though.
Re:NY Times copyright suit wants OpenAI to delete all GPT in... by ml-anon on Thursday December 28, 2023 @07:40PM (#38800501)
Hebbian learning and backprop are not comparable and they don’t have a similar effect in any meaningful sense.