News for hackers. Stuff that matters.

A comment on A modern self-referential weight matrix that learns to modify itself. Parent comment

Re:A modern self-referential weight matrix that learns to mo... by erdos4d on Wednesday April 13, 2022 @03:58PM (#31019505)
Even if the algorithms were here to do that job, where will the hardware come from? I'm staring at almost a decade of commercial processor (i7) lineage right now and the jump has been from 4 to 6 cores with no change in clock speed (maybe the newer one is even slower actually). There definitely won't be any singularity jumping off unless today's hardware gets another few decades of Moore's law, and that is not happening.
[ Parent | on HN ]
Re:A modern self-referential weight matrix that learns to mo... by beebeepka on Wednesday April 13, 2022 @07:35PM (#31021642)
Greetings, time traveller. Welcome to 2022! We've had 16-cire consumer CPUs for almost half a decade. And even higher core counts on the higher end platforms. But you already knew all that, didn't you
[ Parent | on HN ]
Re:A modern self-referential weight matrix that learns to mo... by dotnet00 on Wednesday April 13, 2022 @06:35PM (#31021145)
It's a bit dishonest to be looking at specifically the manufacturer who spent most of the past decade enjoying its effective monopoly on desktop CPUs as a reference for how computers have improved. Even moreso since 4 and 6 core CPUs are not representative of the high end systems used to train even current state of the art ML models.
[ Parent | on HN ]
Re:A modern self-referential weight matrix that learns to mo... by armchair_ on Wednesday April 13, 2022 @04:18PM (#31019735)
Well, the code that was published alongside the article is written in Python and CUDA, so you're not looking at the right kind of processor to start.

My 5-year-old, consumer-grade GPU does 1.5 MHz * 2300 cores, whereas the equivalent released this year does 1.7 Mhz * 8900 cores. Granted, not the best way to measure GPU performance, but it is roughly keeping pace with Moore's law, and it's going to be a better indicator of the future than Intel CPU capabilities, especially for machine learning applications.

[ Parent | on HN ]
Re:A modern self-referential weight matrix that learns to mo... by erdos4d on Wednesday April 13, 2022 @04:38PM (#31019936)
So you are saying that we can get another 3 decades of Moore's law by switching to GPUs made of silicon rather than CPUs made of silicon? Well fuck, problem solved then. I was completely unaware it was so easy.
[ Parent | on HN ]
Re:A modern self-referential weight matrix that learns to mo... by VyperCard on Wednesday April 13, 2022 @06:33PM (#31021139)
Yeah, but mostly for matrix multiplications
[ Parent | on HN ]
Re:A modern self-referential weight matrix that learns to mo... by ponow on Thursday April 14, 2022 @04:52AM (#31024635)
GHz not Mhz for your GPU specs.
[ Parent | on HN ]
Re:A modern self-referential weight matrix that learns to mo... by p1necone on Thursday April 14, 2022 @12:00AM (#31023272)
I'll give you the benefit of the doubt and assume you just haven't looked at the CPU market in the last 5 years, but AMD kicked Intels ass, and Intel is also finally being forced to compete now. There's even a 16c/32t CPU on AMDs standard consumer socket: https://www.amd.com/en/products/cpu/amd-ryzen-9-5950x
[ Parent | on HN ]