News for hackers. Stuff that matters.
Re:ViperGPT: A Python Framework That Combines Vision and Lan... by vosper on Friday March 17, 2023 @01:51PM (#35200419)
Is there read really a Python library called ImagePatch that can find any item in an image, and it works as well as in this video? Google didn’t find an obvious match for “Python ImagePatch”
Re:ViperGPT: A Python Framework That Combines Vision and Lan... by SCUSKU on Friday March 17, 2023 @02:03PM (#35200600)
Looks like they haven't released their code yet, but my guess is that it's an in house wrapper around CLIP or something similar?
Re:ViperGPT: A Python Framework That Combines Vision and Lan... by leobg on Friday March 17, 2023 @03:12PM (#35201533)
I guess the idea is to trick the model into generating pseudo code. Which really doesn’t do much more than to act as a “scratchpad“ to focus the attention of the model to reason through the problem.

Besides, the Codex models are free right now. So… one more reason to rephrase questions as coding questions ;-)

Re:ViperGPT: A Python Framework That Combines Vision and Lan... by vosper on Friday March 17, 2023 @05:40PM (#35203276)
Oh, so maybe I misunderstood what I was seeing. It wrote pseudo-code that makes sense conceptually, not code that I can paste in Jupyter and run (given the right imports)?

That sure wasn't obvious from the video.

Re:ViperGPT: A Python Framework That Combines Vision and Lan... by og_kalu on Friday March 17, 2023 @09:36PM (#35205176)
It's not actully pseudocode. If you read the paper, these are functions/libraries they introduce that haven't been published to github yet.
Re:ViperGPT: A Python Framework That Combines Vision and Lan... by og_kalu on Saturday March 18, 2023 @12:02AM (#35205946)
It's not actully pseudocode. If you read the paper, these are functions/libraries they introduce that haven't been published to github yet.
Re:ViperGPT: A Python Framework That Combines Vision and Lan... by leobg on Friday March 17, 2023 @03:14PM (#35201551)
There is a GitHub repo / Python lib called com2fun which exploits this. Allows you to get results from functions that you only pretend exist. (Am on mobile and can’t link to it right now.)
Re:ViperGPT: A Python Framework That Combines Vision and Lan... by make3 on Friday March 17, 2023 @05:16PM (#35203021)
it's just a separate vision model. you just have to use a state of the art instance segmentation model, the task shown are really not that hard.

it's not "just a library"

Re:ViperGPT: A Python Framework That Combines Vision and Lan... by vosper on Friday March 17, 2023 @05:38PM (#35203265)
So the code that was written by the AI in the video doesn't actually work as written?
Re:ViperGPT: A Python Framework That Combines Vision and Lan... by og_kalu on Friday March 17, 2023 @09:35PM (#35205172)
It does. If you red the paper, these are functions/libraries they introduce that haven't been published to github yet.
Re:ViperGPT: A Python Framework That Combines Vision and Lan... by VectorLock on Friday March 17, 2023 @10:41PM (#35205526)
Almost as interesting as the GPT part of it.
Re:ViperGPT: A Python Framework That Combines Vision and Lan... by drothlis on Monday March 20, 2023 @07:50AM (#35230200)
According to the ViperGPT paper their "ImagePatch.find()" uses GLIP.

According to the GLIP paper,† accuracy on a test-set not seen during training is around 60% so... neat demos but whether it'll be reliable enough depends on your application.

† https://arxiv.org/abs/2206.05836