News for hackers. Stuff that matters.

A comment on Gemma 3 QAT Models: Bringing AI to Consumer GPUs. Parent comment

Re:Gemma 3 QAT Models: Bringing AI to Consumer GPUs by ein0p on Sunday April 20, 2025 @03:10PM (#43745820)
No, tagging of people is already handled by another model. Gemma just describes what's in the image, and produces a comma separated list of keywords. No additional training is required besides a few tweaks to the prompt so that it outputs just the description, without any "fluff". E.g. it normally prepends such outputs with "Here's a description of the image:" unless you really insist that it should output only the description. I suppose I could use constrained decoding into JSON or something to achieve the same, but I didn't mess with that.

On some images where Gemma3 struggles Mistral Small produces better descriptions, BTW. But it seems harder to make it follow my instructions exactly.

I'm looking forward to the day when I can also do this with videos, a lot of which I also have no interest in uploading to someone else's computer.

[ Parent | on HN ]
Re:Gemma 3 QAT Models: Bringing AI to Consumer GPUs by fer on Sunday April 20, 2025 @04:30PM (#43746325)
How do you use the keywords after? I have Immich running which does some analysis, but the querying is a bit of a hit and miss.
[ Parent | on HN ]
Re:Gemma 3 QAT Models: Bringing AI to Consumer GPUs by ein0p on Sunday April 20, 2025 @04:44PM (#43746405)
Search is indeed hit and miss. Immich, for instance, currently does absolutely nothing with the EXIF "description" field, so I store textual descriptions on the side as well. I have found Immich's search by image embeddings to be pretty weak at recall, and even weaker at ranking. IIRC Lightroom Classic (which I also use, but haven't found a way to automate this for without writing an extension) does search that field, but ranking is a bit of a dumpster fire, so your best bet is searching uncommon terms or constraining search by metadata (e.g. not just "black kitten" but "black kitten AND 2025"). I expect this to improve significantly over time - it's a fairly obvious thing to add given the available tech.
[ Parent | on HN ]
Re:Gemma 3 QAT Models: Bringing AI to Consumer GPUs by ethersteeds on Monday April 21, 2025 @01:02AM (#43748621)
> No, tagging of people is already handled by another model.

As an aside, what model/tools do you prefer for tagging people?

[ Parent | on HN ]
Re:Gemma 3 QAT Models: Bringing AI to Consumer GPUs by mentalgear on Monday April 21, 2025 @02:18AM (#43748921)
Since you already seem to have done some impressive work on this for your personal use, would you mind open sourcing it?
[ Parent | on HN ]