Re:ViperGPT: A Python Framework That Combines Vision and Lan...
by vosper on Friday March 17, 2023 @01:51PM
(#35200419)
Is there read really a Python library called ImagePatch that can find any item in an image, and it works as well as in this video? Google didn’t find an obvious match for “Python ImagePatch”
[ Parent
| on HN ]
Re:ViperGPT: A Python Framework That Combines Vision and Lan...
by SCUSKU on Friday March 17, 2023 @02:03PM
(#35200600)
Looks like they haven't released their code yet, but my guess is that it's an in house wrapper around CLIP or something similar?
Re:ViperGPT: A Python Framework That Combines Vision and Lan...
by leobg on Friday March 17, 2023 @03:12PM
(#35201533)
I guess the idea is to trick the model into generating pseudo code. Which really doesn’t do much more than to act as a “scratchpad“ to focus the attention of the model to reason through the problem.
Besides, the Codex models are free right now. So… one more reason to rephrase questions as coding questions ;-)
Re:ViperGPT: A Python Framework That Combines Vision and Lan...
by vosper on Friday March 17, 2023 @05:40PM
(#35203276)
Oh, so maybe I misunderstood what I was seeing. It wrote pseudo-code that makes sense conceptually, not code that I can paste in Jupyter and run (given the right imports)?
That sure wasn't obvious from the video.
Re:ViperGPT: A Python Framework That Combines Vision and Lan...
by og_kalu on Friday March 17, 2023 @09:36PM
(#35205176)
It's not actully pseudocode. If you read the paper, these are functions/libraries they introduce that haven't been published to github yet.
Re:ViperGPT: A Python Framework That Combines Vision and Lan...
by og_kalu on Saturday March 18, 2023 @12:02AM
(#35205946)
It's not actully pseudocode. If you read the paper, these are functions/libraries they introduce that haven't been published to github yet.
Re:ViperGPT: A Python Framework That Combines Vision and Lan...
by leobg on Friday March 17, 2023 @03:14PM
(#35201551)
There is a GitHub repo / Python lib called com2fun which exploits this. Allows you to get results from functions that you only pretend exist. (Am on mobile and can’t link to it right now.)
Re:ViperGPT: A Python Framework That Combines Vision and Lan...
by make3 on Friday March 17, 2023 @05:16PM
(#35203021)
it's just a separate vision model. you just have to use a state of the art instance segmentation model, the task shown are really not that hard.
it's not "just a library"
Re:ViperGPT: A Python Framework That Combines Vision and Lan...
by vosper on Friday March 17, 2023 @05:38PM
(#35203265)
So the code that was written by the AI in the video doesn't actually work as written?
Re:ViperGPT: A Python Framework That Combines Vision and Lan...
by og_kalu on Friday March 17, 2023 @09:35PM
(#35205172)
It does. If you red the paper, these are functions/libraries they introduce that haven't been published to github yet.
Re:ViperGPT: A Python Framework That Combines Vision and Lan...
by VectorLock on Friday March 17, 2023 @10:41PM
(#35205526)
Almost as interesting as the GPT part of it.
Re:ViperGPT: A Python Framework That Combines Vision and Lan...
by drothlis on Monday March 20, 2023 @07:50AM
(#35230200)
According to the ViperGPT paper their "ImagePatch.find()" uses GLIP.
According to the GLIP paper,† accuracy on a test-set not seen during training is around 60% so... neat demos but whether it'll be reliable enough depends on your application.