News for hackers. Stuff that matters.
Re:Using GPT-4 Vision with Vimium to browse the web by snake_doc on Thursday November 09, 2023 @12:03AM (#38201360)
Ah, very similar to Adept’s[1] concept? Though, their product seems not yet ready.

[1] https://www.adept.ai/

Re:Using GPT-4 Vision with Vimium to browse the web by karmasimida on Thursday November 09, 2023 @12:22AM (#38201475)
This is precisely the demo I am thinking.
Re:Using GPT-4 Vision with Vimium to browse the web by ishan0102 on Thursday November 09, 2023 @12:15AM (#38201434)
Yep, took inspiration from them and a couple other startups
Re:Using GPT-4 Vision with Vimium to browse the web by QkPrsMizkYvt on Thursday November 09, 2023 @05:00AM (#38203057)
What other startups did you use for inspiration?
Re:Using GPT-4 Vision with Vimium to browse the web by jatins on Thursday November 09, 2023 @02:22AM (#38202120)
It's also a little insane to me that what Adept has been supposedly building for years with 300+ mil in funding can now be built in a day with Open AI APIs?

I think Adept pivoted along the way but original concept was very similar to this.

Re:Using GPT-4 Vision with Vimium to browse the web by sunshadow on Thursday November 09, 2023 @06:10AM (#38203550)
But its too expensive to become practical with the OpenAI API. Also, demo is cool until you see the real-world webpages, then you'll realize that this only works less than %50 of webpages.
Re:Using GPT-4 Vision with Vimium to browse the web by og_kalu on Thursday November 09, 2023 @07:34AM (#38204192)
GPT-4V may be surprisingly robust here. Set of mark prompting(which is accomplished here with Vim) improves grounding by a silly high amount. https://som-gpt4v.github.io/
Re:Using GPT-4 Vision with Vimium to browse the web by abrichr on Friday November 10, 2023 @07:49AM (#38218194)
Agreed! This is part of the motivation behind https://github.com/OpenAdaptAI/OpenAdapt
Re:Using GPT-4 Vision with Vimium to browse the web by amks on Thursday November 09, 2023 @02:26PM (#38209698)