News for hackers. Stuff that matters.
Re:Imagen, a text-to-image diffusion model by james-redwood on Monday May 23, 2022 @05:17PM (#31484867)
Metacalculus, a mass forecasting site, has steadily brought forward the prediction date for a weakly general AI. Jaw-dropping advances like this, only increase my confidence in this prediction. "The future is now, old man."

https://www.metaculus.com/questions/3479/date-weakly-general...

Re:Imagen, a text-to-image diffusion model by tpmx on Monday May 23, 2022 @05:46PM (#31485177)
I don't see how this gets us (much) closer to general AI. Where is the reasoning?
Re:Imagen, a text-to-image diffusion model by quirino on Monday May 23, 2022 @06:32PM (#31485671)
I think this serves at least as a clear demonstration of how advanced the current state of AI is. I had played with GPT-3 and that was very impressive but I couldn't even dream something as good as D-ALLE 2 was already possible.
Re:Imagen, a text-to-image diffusion model by 6gvONxR4sf7o on Monday May 23, 2022 @06:34PM (#31485686)
Big pretrained models are good enough now that we can pipe them together in really cool ways and our representations of text and images seem to capture what we “mean.”
Re:Imagen, a text-to-image diffusion model by tpmx on Monday May 23, 2022 @06:36PM (#31485702)
Yeah, it seems like it. But it's still just complicated statistical models. Again, where is the reasoning?
Re:Imagen, a text-to-image diffusion model by marvin on Monday May 23, 2022 @06:55PM (#31485872)
I still think we're missing some fundamental insights on how layered planning/forecasting/deducting/reasoning works, and that figuring this out will be necessary in order to create AI that we could say "reasons".

But with the recent advances/demonstrations, it seems more likely today than in 2019 that our current computational resources are sufficient to perform magnificantly spooky stuff if they're used correctly. They are doing that already already, and that's without deliberately making the software do anything except draw from a vast pool of examples.

I think it's reasonable, based on this, to update one's expectations of what we'd be able to do if we figured out ways of doing things that aren't based on first seeing a hundred million examples of what we want the computer to do.

Things that do this can obviously exist, we are living examples. Does figuring it out seem likely to be many decades away?

Re:Imagen, a text-to-image diffusion model by tpmx on Monday May 23, 2022 @07:10PM (#31485989)
That's a well-balanced response that I can agree with.

I'm an not AGI-skeptic. I'm just a bit skeptical that the topic of this thread is the path forward. It seems to me like an exotic detour.

And, of course intelligence isn't magic. We're producing new intelligent entities at rate of a about ~5 per second globally, every day.

> Does figuring it out seem likely to be many decades away?

1-7?

Re:Imagen, a text-to-image diffusion model by londons_explore on Monday May 23, 2022 @06:58PM (#31485886)
All it takes is one 'trick' to give these models the ability to do reasoning.

Like for example the discovery that language models get far better at answering complex questions if asked to show their working step by step with chain of thought reasoning as in page 19 of the PaLM paper [1]. Worth checking out the explanations of novel jokes on page 38 of the same paper. While it is, like you say, all statistics, if it's indistinguishable from valid reasoning, then perhaps it doesn't matter.

[1]: https://arxiv.org/pdf/2204.02311.pdf

Re:Imagen, a text-to-image diffusion model by 6gvONxR4sf7o on Monday May 23, 2022 @06:47PM (#31485810)
I don’t care whether it reasons its way from “3 teddy bears below 7 flamingos” to a picture of that or if it gets there some other way.

But also, some of the magic in having good enough pretrained representations is that you don’t need to train them further for downstream tasks, which means non-differentiable tasks like logic could soon become more tenable.

Re:Imagen, a text-to-image diffusion model by renewiltord on Monday May 23, 2022 @06:51PM (#31485843)
A belief oft shared is that sufficiently complicated statistical models are indistinguishable from reasoning.
Re:Imagen, a text-to-image diffusion model by _joel on Monday May 23, 2022 @06:14PM (#31485493)
Perhaps the confluence of NLP and something generative?
Re:Imagen, a text-to-image diffusion model by astrange on Monday May 23, 2022 @06:34PM (#31485689)
That doesn’t even lead in the direction of an AGI. The larger and more expensive a model is the less like an “AGI” it is - an independent agent would be able to learn online for free, not need millions in TPU credits to learn what color an apple is.
Re:Imagen, a text-to-image diffusion model by SemanticStrengh on Monday May 23, 2022 @06:35PM (#31485698)
Yes metaculus mostly bet a magic number based on perhaps and tbh why not, the interaction of NLP and vision is mysterious and has potential. However those magic numbers should still be considered magic numbers. I agree that in 2040 the interactions will have extensively been studied though but the conclusion of wether we czn go much further on cross-models synergies is totally unknown or pessimist.
Re:Imagen, a text-to-image diffusion model by chias on Tuesday May 24, 2022 @12:02AM (#31487847)
"The future is already here — It’s just not very evenly distributed"
Re:Imagen, a text-to-image diffusion model by sydthrowaway on Monday May 23, 2022 @05:21PM (#31484896)
How can we prepare for this?

This will result in mass social unrest.

Re:Imagen, a text-to-image diffusion model by dougmwne on Monday May 23, 2022 @07:01PM (#31485916)
I think the serious answer is that it is yet another labor multiplier like electricity and software. Our tech since the industrial revolution has allowed us to elevate ourselves from a largely agrarian society to space and cyberspace. AI, by all appearances, continues to be a tool, just the latest in a long line of better tools. It still requires a human to provide intent and direction. Right now in my job, I command the collect output of a million medieval scribes. In the future I will command a million Michelangelos.

Should ML/AI deliver on the wildest promises, it will be like a SpaceX Starship for the mind.

Re:Imagen, a text-to-image diffusion model by sydthrowaway on Monday May 23, 2022 @09:00PM (#31486802)
Well, anyone over 40 will be fucked. There goes your utopia.
Re:Imagen, a text-to-image diffusion model by dougmwne on Tuesday May 24, 2022 @06:41AM (#31490304)
Computers didn't fuck anyone over 40, but they did create new opportunities for young people that slowly took over the labor market and provided a steady stream of productivity growth. Right now these are impressive benchmarks and neat toys that cost millions to train. This is going to be a slow transition to a new paradigm. We are not going to end up in a utopia any more than computers created a utopia.
Re:Imagen, a text-to-image diffusion model by machiaweliczny on Tuesday May 24, 2022 @08:06AM (#31490933)
No because once this is live, creating private (teaching) assistants and good UX will be cheaper.
Re:Imagen, a text-to-image diffusion model by refulgentis on Monday May 23, 2022 @05:23PM (#31484922)
You think so? I'm very high on the Kool-Aid, image generation and text transformation models are core parts of my workflow. (Midjourney, GPT-3)

It's still an unruly 7 year old at best. Results need to be verified. Prompt engineering and a sense of creativity are core competencies.

Re:Imagen, a text-to-image diffusion model by visarga on Monday May 23, 2022 @05:39PM (#31485107)
> Prompt engineering and a sense of creativity are core competencies.

It's funny that people are also prompting each other. Parents, friends, teachers, doctors, priests, politicians, managers and marketers are all prompting (advising) us to trigger desired behaviour. Powerful stuff - having a large model and knowing how to prompt it.

Re:Imagen, a text-to-image diffusion model by aaaaaaaaaaab on Monday May 23, 2022 @05:56PM (#31485290)
Stock up on guns, ammo, cigarettes, water filters, canned food, and toilet paper.
Re:Imagen, a text-to-image diffusion model by boppo1 on Monday May 23, 2022 @06:05PM (#31485384)
Nah, learn Spanish and first-aid. Being able to fix people is more useful than having commodities that will make you a target.