News for hackers. Stuff that matters.
Re:AI training is outpacing Moore’s Law by visarga on Wednesday December 08, 2021 @02:37PM (#29489359)
These large models are going to be the operating system of the future. When they are going to run efficiently on edge we are going to see an explosion of automation. Being so large they are not only more accurate, but also quicker learners. So less initial effort to start a project.

Google is teasing us with leaks about Pathways[1], but not releasing a paper or demo yet.

[1] https://blog.google/technology/ai/introducing-pathways-next-...

I call them multi-multi-models: multi-modality, multi-task, multi-language, multi-memory (besides the processing memory it can have search engine in the loop and episodic memory). Somebody's going to make it in the next couple of years.

Re:AI training is outpacing Moore’s Law by andreyk on Wednesday December 08, 2021 @03:17PM (#29489896)
IMHO (as a AI researcher) 'multi-modality, multi-task, multi-language, multi-memory' may take longer than you think. Doing multi modal learning is quite hard due to lack of labeled data, a lot of tasks either don't offer clean plentiful datasets or require RL / human in the loop learning and not just supervised or self supervised learning, and on and on. Certainly a lot of progress will be made and partial automation is likely to speed up (as has happened with Copilot), but full automation seems to be a good while away.

Plug - I expand on this a bit in this blog post 'The Inherent Limitations of GPT-3' (which was on front page of HN a few weeks back and led to a good deal of discussion ) https://lastweekin.ai/p/the-inherent-limitations-of-gpt-3

Re:AI training is outpacing Moore’s Law by Moru on Wednesday December 08, 2021 @02:54PM (#29489587)
The sad part is that we are all helping to train the futures AI but who will own it and make money on it?
Re:AI training is outpacing Moore’s Law by visarga on Friday December 10, 2021 @08:22AM (#29509506)
> who will own it and make money on it?

The nice part about it is that everyone will have access to great models for direct use, fine-tuning and adaptation. But if people don't take this freedom and are content with simply consuming applications, then they depend on those who deploy the apps.

Re:AI training is outpacing Moore’s Law by wetpaws on Wednesday December 08, 2021 @03:05PM (#29489743)
You can have ownership in companies that do machine learning. It's called "shares". Creating your own business is an option too.
Re:AI training is outpacing Moore’s Law by frenchyatwork on Wednesday December 08, 2021 @03:56PM (#29490387)
The complaint was that it's the people who own the equipment (or shares in it), and not the people who do the work, who are reaping the profit.

When the working class had this problem a couple centuries ago, it was solved by building guillotines. I hope we can do better this time.

Re:AI training is outpacing Moore’s Law by thehappypm on Wednesday December 08, 2021 @05:03PM (#29491147)
You’re conflating two things. Feudalism is like slavery: taking credit for someone else’s labor. Automation removes the need for labor. Automation has almost universally led to quality of life improvements, not revolution.
Re:AI training is outpacing Moore’s Law by rowanG077 on Wednesday December 08, 2021 @04:06PM (#29490503)
To start an AI company you generally don't need much. A few thousands in GPUs and you are good to go. If AI researchers feel they are underpaid there is literally nothing stopping them from starting a company themselves.
Re:AI training is outpacing Moore’s Law by gmadsen on Wednesday December 08, 2021 @04:46PM (#29490959)
that is absolutely not true. I'm not sure where you are getting your information, but read anything from an angel investor. most AI start ups fail because they go into huge debt getting correct data and training models, which end up bankrupting them in the process with nothing to show for it
Re:AI training is outpacing Moore’s Law by rowanG077 on Thursday December 09, 2021 @07:27AM (#29496579)
If you need to collect date then you are not only an AI company. And in that case a large company is indeed needed.
Re:AI training is outpacing Moore’s Law by wetpaws on Wednesday December 08, 2021 @05:43PM (#29491542)
I don't see any problem with it, stock market is open for everyone.
Re:AI training is outpacing Moore’s Law by giovannibonetti on Wednesday December 08, 2021 @07:06PM (#29492288)
Nowadays startups often delay going to the stock market, and remain private while they're growing fast. It's only when they are growing more slowly and the VCs want to get out that they go public.

This way, the average investor usually does not reap outsized benefits from the ride.