News for hackers. Stuff that matters.

A comment on Show HN: LLaVaVision: An AI "Be My Eyes"-like web app with a llama.cpp backend.

Re:Show HN: LLaVaVision: An AI "Be My Eyes"-like web app wit... by enoch2090 on Monday November 06, 2023 @01:47AM (#38159572)
This is cool! One thing that can be easily improved is that the narrator is still trying to describe everything, while people using Be My Eyes they usually got specific purposes (or "attention"). Would be nice to also implement whisper.cpp and use query inputs with some CoT or prompting to refine the description.
[ Parent | on HN ]
Re:Show HN: LLaVaVision: An AI "Be My Eyes"-like web app wit... by lxe on Monday November 06, 2023 @02:05AM (#38159645)
This is an awesome idea! Yeah refining and asking more questions about the image could make it viable. The BakLlava model is pretty impressive, but I do need to tune the prompts and the hyperparams as well. Or even do a small finetune. It’s a fun space to dive into.
[ Parent | on HN ]
Re:Show HN: LLaVaVision: An AI "Be My Eyes"-like web app wit... by umtksa on Monday November 06, 2023 @05:45AM (#38160951)
awesome, love the idea and maybe putting a default prompt like:

"this images comes from the persons POV that ask you questions"

instead of "a person holding something" "you are holding xyz" wold be better

[ Parent | on HN ]