News for hackers. Stuff that matters.
Re:Vector Databases: A Technical Primer [pdf] by yagami_takayuki on Friday January 12, 2024 @03:05PM (#38973244)
For recognizing features such as hair and skin color, which would do a better job? Machine learning with image classification? Or a vector database?

I've had Weaviate return a dog given a human as input when doing an image similarity search, so I was wondering if there's some way to improve the results or whether I'm barking up the wrong tree.

Re:Vector Databases: A Technical Primer [pdf] by dimatura on Friday January 12, 2024 @03:12PM (#38973327)
For those two in particular? You'd definitely get a better result with an ML model such as a convolutional neural net. In some sense, using an image similarity query is a kind of ML model - nearest neighbor - which can work in some scenarios. But for this specifically, I'd recommend a CNN.
Re:Vector Databases: A Technical Primer [pdf] by yagami_takayuki on Friday January 12, 2024 @06:00PM (#38975333)
Thanks will experiment with a CNN.
Re:Vector Databases: A Technical Primer [pdf] by bfeynman on Friday January 12, 2024 @03:12PM (#38973319)
I would think you could improve your embedding space to address that issue, partially. Similarity search (as a result of some contrastive loss) definitely suffers at the tails and the OOD is pretty bad. That being said, you're more likely to have higher recall than a more classical technique.
Re:Vector Databases: A Technical Primer [pdf] by yagami_takayuki on Friday January 12, 2024 @06:00PM (#38975335)
Thanks for your answer!
Re:Vector Databases: A Technical Primer [pdf] by m00x on Friday January 12, 2024 @05:33PM (#38975074)
You don't use vector databases independently, you need to input the embedding from a ML model.

For your use-case, it should be pretty simple. You could use a CNN and train it, or use YOLO, Deepface, or other face detection algos, then within the face, find the hair and find the skin.

From there you can use a vector database to get the colors that resemble other inputs, or you can use a simple CNN to classify the hair and skin to the closest label.

Re:Vector Databases: A Technical Primer [pdf] by yagami_takayuki on Friday January 12, 2024 @06:00PM (#38975330)
Thanks! Will looks into this.
Re:Vector Databases: A Technical Primer [pdf] by Kubuxu on Friday January 12, 2024 @06:10PM (#38975426)
You could consider something like BLIP2. There are multiple ways you could use it: embed images and match them against embeddings of text descriptors, train custom embedding of descriptors, or a classifier on top of the embedding (linear layer on top of the image embedding network).

The approaches increase in complexity. It also allows for dataset bootstrap:

Let's say you want to classify cats by breed. You could start by embedding images and text descriptors and distance-matching the embedded descriptors to the images. This gives you a dataset that might be 90% correct; you can then clean it up, which would be easier to do than manually labelling it. Based on that improved dataset, you can train a custom embedding for the labels or a classification layer on top of the image embedding network.

Re:Vector Databases: A Technical Primer [pdf] by yagami_takayuki on Friday January 12, 2024 @07:15PM (#38975947)
thank you, will look into BLIP2.