"Moving fast" may take on a whole new meaning and I'd put money on the rate of iteration soon being beyond the vast majority's comprehension (myself included).

it's already beyond my comprehension, i've not lived very long but in the time i have i've never seen any technology develop so rapidly and at such a rapidly increasing pace. I assume this is what it must have felt like during the dawn of the age of computing.

It makes you wonder if those singularity proponents don't have a point, and it all depends on whether it keeps accelerating or whether it will slow down again. I hope for the latter and I fear for the former. Even if it does slow down eventually a long enough period of such change is going to make the industrial revolution (whose negative effects we are still coming to terms with today!) like a walk in the park.

Unloading Ray Kurzweil to the cloud in 5, 4, 3…

Reality is biased towards the fast-moving scenario, so long as we aren’t running into the bounds of physics, which as far as I can tell we’re not. Kurzweil was much more right than he was wrong. The opposite is true of people who strongly disagreed with him and called him a quack.

The transhumanist movement has more than its fair share of quacks, but I think Kurzweil is enough of a scientist to take his arguments a bit more serious. That said, I'm getting pretty tired of all the mind uploading, eternal life and other afterlife nonsense. That to me is just religion in a new jacket.

Ya there isn’t enough talk of the more medium-term, practical uses of AI. I don’t care about mind uploading or AI doom risk. Will AI make stuff cheaper and better? Will it incrementally improve my life? I think the answers are yes and yes. That’s where the focus should be. Where and how can AI can help in construction, finance, medicine, etc…

A so-called singularity would require accelerated development for many more technological spheres, not just semiconductor fabrication, and related information computation and AI advances. Logistics, supply chains, mining, farming, manufacturing, energy, biotechnology. While the former may continue to accelerate development in the latter categories, the scale of such impact is purely speculative. I don't believe the advances will be proportional in these harder spheres, as their physicality cannot be as readily manipulated as information.

Advances in the harder spheres will come. It’s just a matter of time. Their transformation will happen in a step-wise fashion, unlike the curvy exponential growth you’re seeing in pure software.

AI cannot safely and cheaply be used to drive trucks. But as soon as it can with one truck, it can with millions of trucks. All at once.

AI and robots haven’t advanced enough to replace construction workers for fluid, dexterous tasks, but as soon as they do, robots can replace millions of construction workers and surpass them in sophistication and speed.

This will happen in our lifetime, and the change will be extremely transformative.

Material science and genetics are prime candidates for a different approach and AI techniques are already paying off dividends in those domains.

We're already in the phase of AI self improvement, albeit through human mediation for the time being with copilot and other code generation tools that are used to get the next improved version out faster.

1.5 years is actually not that bad. In fact, all changes and improvements to LLMs since the original Transformer paper is just the size -- tensor dimension, layers, etc. GPT-3, which is still widely used today, was proposed more than 3 years ago.

Seems to align with Google, seems like they release a new TPU every 1.5 to 2 years.

I think modern LLMs are powerful enough now that they will still be useful in a couple years even if they aren't state-of-the-art. ChatGPT still lets you run their older model for cheaper than running GPT4, I could see a world where GPT4 is still available in 1.5 years even if there are better models out there.