News for hackers. Stuff that matters.
Re:Vector Databases: A Technical Primer [pdf] by Kubuxu on Friday January 12, 2024 @06:10PM (#38975426)
You could consider something like BLIP2. There are multiple ways you could use it: embed images and match them against embeddings of text descriptors, train custom embedding of descriptors, or a classifier on top of the embedding (linear layer on top of the image embedding network).

The approaches increase in complexity. It also allows for dataset bootstrap:

Let's say you want to classify cats by breed. You could start by embedding images and text descriptors and distance-matching the embedded descriptors to the images. This gives you a dataset that might be 90% correct; you can then clean it up, which would be easier to do than manually labelling it. Based on that improved dataset, you can train a custom embedding for the labels or a classification layer on top of the image embedding network.

Re:Vector Databases: A Technical Primer [pdf] by yagami_takayuki on Friday January 12, 2024 @07:15PM (#38975947)
thank you, will look into BLIP2.