News for hackers. Stuff that matters.
Re:Meta's Segment Anything written with C++ / GGML by dekhn on Tuesday September 05, 2023 @07:44PM (#37399391)
Do you know how the time to do the image embedding takes? In SAM, most of the time is spent generating a very expensive embedding (prohibitive for real-time object detection). From the timing on your page it looks like yours is also similarly slow, but I'm curious how it compares to the pytorch Meta implementation.
Re:Meta's Segment Anything written with C++ / GGML by yavorgiv on Wednesday September 06, 2023 @07:47AM (#37403743)
I am the creator of the repo.

Depends on the machine, number of threads selected and the model checkpoint used (Vit-B or Vit-L or Vit-B). The video demo attached is running on Apple M2 Ultra and using the Vit-B model. The generation of the image embedding takes ~1.9s there and all the subsequent mask segmentations take ~45ms.

However, I am now focusing on improving the inference speed by making better use of ggml and trying out quantization. Once I make some progress in this direction I will compare to other SAM alternatives and benchmark more thoroughly.

Re:Meta's Segment Anything written with C++ / GGML by billrobertson42 on Wednesday September 06, 2023 @09:27AM (#37404803)
This is amazing. Thank you!