News for hackers. Stuff that matters.

Show HN: Semantic Video Search

https://ramanlabs.in/static/videointel.html

Hi HN, I am Anubhav from Ramanlabs. We have been working on a native gui application to allow users to search any video data( mp4, mkv) or video streams (http/rtsp) using computer vision.

Application is supposed to work like a video player which displays decoded frames and recognizes objects concurrently, making it an interactive experience. It works in super real-time and only expects a quad-core CPU with AVX2 instructions at minimum.

Application is free to download (without any signup/account). We are only supporting WINDOWS for now [0]. Even though this is a binary application, we have ZERO telemetry/analytics builtin.

User interface is limited for now, and definitely needs more work. But we are releasing it here for early feedback/bugs.

We would love for you to try it out and hear your thoughts/feedback.

[0] We should also have a linux version ready in few days.

( 16 comments | 2 points | on HN )
Re:Show HN: Semantic Video Search by daaayum on Thursday November 17, 2022 @09:56AM (#33639680)
This is awesome. Would be great is i could use this across multiple video files and search in a library of files.
Re:Show HN: Semantic Video Search by warangal on Thursday November 17, 2022 @11:42AM (#33641504)
Still a lot of work pending on user interface side, but this is definitely on the TODO list. We are also working to allow users to search for a lot more categories of objects in the upcoming releases. It has been a lot of work, so thanks for feedback and checking out!
Re:Show HN: Semantic Video Search by yboris on Thursday November 17, 2022 @11:26PM (#33650290)
This is the feature I'll be building this winter into my Video Hub App

https://videohubapp.com/ && https://github.com/whyboris/Video-Hub-App

I have a branch that detects faces and extracts vectors, it's just a matter of aggregating them and creating a UI so users can search for videos by face (or by dropping an image with a face).

Re:Show HN: Semantic Video Search by metadat on Thursday November 17, 2022 @11:20AM (#33641119)
Please post again when the Linux version is out!

Will also be great if you create a command-line version.

Re:Show HN: Semantic Video Search by warangal on Thursday November 17, 2022 @11:32AM (#33641325)
Thanks for checking out. We are also working on SDK to make it easier to integrate with existing applications. Linux version is almost ready, just looking at ways to distribute dependencies a bit more easily.
Re:Show HN: Semantic Video Search by rch on Thursday November 17, 2022 @01:13PM (#33643121)
Nixpkg, w/ allowUnfree.

If you try it I'd be interested in learning about your experience, as I'm looking into doing something similar early next year (distribution, not similar to the app itself).

Re:Show HN: Semantic Video Search by warangal on Thursday November 17, 2022 @10:54PM (#33649990)
Thanks for suggestion! Haven't worked with Nix ecosystem yet, but definitely going to check it out and see what works in the end.
Re:Show HN: Semantic Video Search by blooalien on Thursday November 17, 2022 @05:33PM (#33646888)
I personally like AppImage best of the "non-native" app package formats, but ideally you could set up one of the native package build CI pipelines somewhere and automagically build native distro packages from your source tree for the most common three or four native package formats and let the community build more "esoteric" packages for themselves from those possibly?
Re:Show HN: Semantic Video Search by thefourthchime on Thursday November 17, 2022 @02:06PM (#33643941)
Looks pretty cool! I work for a large content company, we would maybe use this if it had:

- A way to run in a container in linux - Or a REST API we could use to run it.

Re:Show HN: Semantic Video Search by warangal on Thursday November 17, 2022 @10:52PM (#33649972)
Yes, we should be able to containerize it easily as we already tested it for virtual linux machine. Is there a way to contact you, in case we add this and more features in the future?
Re:Show HN: Semantic Video Search by sharpshoot on Thursday November 17, 2022 @02:45PM (#33644524)
try www.muse.ai
Re:Show HN: Semantic Video Search by jcims on Thursday November 17, 2022 @04:06PM (#33645708)
I don't know why but this made me wonder if you could train a model on eye tracking data from videos then use that to predict what people would look at with new content.
Re:Show HN: Semantic Video Search by Rodeoclash on Thursday November 17, 2022 @04:32PM (#33646071)
I have an open source video player for esports coaches to review gameplay footage (https://www.vodon.gg/)

Could I use something like this (or a library) to easily recognise enemy players that have shown up in frames? I would love to be able to automatically populate bookmarks of interesting moments in the match.

Re:Show HN: Semantic Video Search by warangal on Thursday November 17, 2022 @11:36PM (#33650379)
Video player looks cool! Even though this is based for badminton, i have found this one <https://cvit.iiit.ac.in/research/projects/cvit-projects/badm...> quite interesting for sport analytics projects.

I think interesting moments bookmarking is more of an open-ended problem and would very much depend on the game, but large computer-vision models like CLIP have proved to be really useful in recognizing general-purposes activities. You could sample frames uniformly from a game video, index them with a text-captioning model, then find/curate a subset of those captions based on your definition of interesting, and then use those curated captions to look for those moments in future videos, i think!

Re:Show HN: Semantic Video Search by Rodeoclash on Friday November 18, 2022 @10:43PM (#33666709)
Hey, that's a really great idea! Thanks, I'm going to play around with it.
Re:Show HN: Semantic Video Search by donqu1xote1 on Thursday November 17, 2022 @11:46PM (#33650466)
This is a great product. Is this one an open-source project or not? Can we collaborate on some features? Take a look at our product here:https://github.com/illacloud/illa-builder