Sharing Screen with GPT 4 vision

Re:Sharing Screen with GPT 4 vision by anonzzzies on Thursday November 09, 2023 @08:16AM (#38204588)

In it's current form, no chance. But if we continue on this trajectory (if that's possible of course), then pretty soon. I personally think there will be a barrier popping up soon for further advancement. However, today I am picking olives (not a joke) for my new career if it should happen (a joke; I would just retire).

[ Parent | on HN ]

Re:Sharing Screen with GPT 4 vision by flir on Thursday November 09, 2023 @09:04AM (#38205115)

You don't think it's already plateaued? In terms of reasoning ability, anyway.

[ Parent | on HN ]

Re:Sharing Screen with GPT 4 vision by anonzzzies on Thursday November 09, 2023 @09:18AM (#38205268)

Well yes, that's the feeling I have, but there are now so many very smart people working on smarter ways and billions are poured in (not only on the 'AI for car searches' type of sites...); I'm willing to give it some benefit of the doubt.

[ Parent | on HN ]

Re:Sharing Screen with GPT 4 vision by kaibee on Thursday November 09, 2023 @11:23AM (#38207066)

Not even close. You can look at the Llama v2 training curves and its clear that even 7B models are still under-trained and they themselves have only been trained on 2T tokens. A 70B or whatever size/model amalgamation GPT-4 is also likely under-trained.

Apart from just plain under-training, there's that whole 'grokking' phenomenon that's been observed in smaller models, where it looks like they're not really improving for a long time, but _eventually_ suddenly undergo a massive improvement. I don't know if anyone has been willing to set enough cash on fire to see if grokking can happen in a large LLM and/or how long it would need to be trained for.

There's also still a lot of juice left in improving the quality of datasets and in training on larger data sets. OpenAI, having built GPT-4, probably want to work on some applications of it given that they have enough of a lead on everyone else. I think there's definitely a few "burn piles of cash to train a better model" buttons at OpenAI right now, but they have no reason to use them when they're clearly in the lead anyway.

There's also the actual limiting factor: there are only so many A100s/H100s in the world, but the amount is growing.

[ Parent | on HN ]

Re:Sharing Screen with GPT 4 vision by Suneel478 on Thursday November 09, 2023 @11:33AM (#38207184)

Yeah, even RL tuning can be applied to improve reasoning paths for specialized applications, something like RLAIF by anthropic but for specialized agents

[ Parent | on HN ]