I use object detection to track tardigrades in my custom motorized microscope. It's very useful for making long observations in a field much larger than the scope's field of view.

The system works quite simply: I start with an existing object detector and train it with a small (<100) number of manually labelled images. Then during inference, I move the scope's field of view using motor commands to put the center of the tardigrade at the center of the field of view.

This technology is very useful for doing long-term observations of tardigrades (so, useful for science).

Thank you! Is the detection accurate enough, or simply the observation conclusions are not that sensitive to minor errors?

That makes me want to revisit my previous idea: boiling soup spillage detector. I once had a google meeting with a cooking soup to keep an eye on it and thought, heck, that seems like a nice exercise for a visual detector finetune

The detection was accurate enough for me to complete one prototype experiment under controlled conditions- a single tardigrade in an otherwise empty field, and even then, it did lose the tardigrade once or twice. Different lighting conditions, and other things in the field like tardigrade eggs, algae and dirt all make it more challenging.

To make it truly ready for production science, I'd need to put more work into making the model robust. I'd also like better object tracking, so I could track multiple unique tardigrades.

If you want to see even better examples, take a look at DeepLabCut, https://www.mackenziemathislab.org/deeplabcut especially the video examples.

I've been very persistent over the past few months in developing a system for agriculture as a primary use case. I want to deploy features to classify crop type, height, vegetation stage, and other important metrics to achieve real-time or near real-time analytics.

Do you have any suggestions on how to proceed further? So far, I've procured a Jetson, five cameras, a stand to fix and calibrate the modules, and a cam array hat to equip four cameras and the jetson. I was checking out VPU and NPUS and other hardware as well but struggling to identify compatible hardware. How can I get ahead and build such model to test and validate in 3 Months of time ?

I'm working on a project that detects climbing holds and lets you set routes by selecting them. (The usual method is putting a bit of colored tape on each hold, where the color corresponds to a route. This works great but becomes difficult to read once more than four or five routes share a hold.) YOLO made the computer vision part of this pretty smooth sailing.

Embed the holds with a led and ir sensor a la swift concert [0] and you’ve got the whole package.

0. https://news.ycombinator.com/item?id=40492515

Frigate uses models like this one for NVR:

https://frigate.video/

As you guessed, high-speed machine vision stuff is frequently used in manufacturing settings for sorting or various quality control tasks. Imagine a picking out bad potatoes on a conveyor belt moving 10s of potatoes per second, or identifying particle counts and size distributions in a stream of water to gauge water quality.

Plus, being a NN it might be possible to detect a foreign object with relative ease (comparing to the classic computer vision); like a rat

Behavior outside of the training distribution is undefined and more often than not undesirable. NNs work well on stuff they’re trained on.

To surveil people in the streets.

Yeah, that's what I'm afraid

You don't need to be afraid, this is already in use in Singapore and China, for example. In the US not so much with persons as far as I know, but certainly with cars.

Field mice that I thought were moles have destroyed my yard. There’s so many tunnels that I can’t tell which are most active. A camera AI that can show which parts of the ground changed significantly would be nice.

At a hotel, we had a problem of luggage carts going missing. There’s a few ways to deal with that. A generic one that would support other, use cases would be to let the camera tell you the last room it went in. Likewise, outdoor cameras might tell you which vehicles had a customer walk in the hotel and which might be non-guests.

Great question! I work for a computer vision company (Roboflow) and have seen computer vision used for everything from accident prevention on critical infrastructure to identifying defects on vehicle parts to detecting trading cards for use in video game applications.

Drawing bounding boxes is a common end point for demos, but for businesses using computer vision there is an entire world after that: on device deployment. This can be on devices like an NVIDIA Jetson (a very common choice), to Raspberry Pis to central CUDA GPU servers for processing large volumes of data (maybe connected to cameras over RTSP).

Note: There are many models that are faster and perform better than YOLOv5 (i.e. YOLOv8, YOLOv10, PaliGemma). Roboflow Inference that our ML team maintains has various guides on deploying models to the edge: https://inference.roboflow.com/#inference-pipeline

Can you go into some examples?

I've used YOLOv5 models in robotics for object detection. While VLMs are great at describing images more generally, having a bounding box of a detection with a confidence score is very useful when paired with depth cameras for locating objects in an open environment. Especially when it can be run on-board and at framerate.

> I've also been a part of the Kinect craze and made 3 demos (games mostly) using their SDK

I ought a Kinect but never got it working correctly with a Mac. What is the state of the art here ? Is there active development anywhere ?

This hailo 8l is extremely useful in mobile robotic systems particularly when paired with an rpi 5. The main board used in this demo is.... not particularly useful however. Generally speaking boards like fpga miss the forest for the trees. As a systems engineer, where am I supposed to put such big board? Also I can expect that, even if my team incorporated this board into a mobile system, it become vaporware well before we deploy anything due to low production numbers and we'd be paying ebay scalpers 2x as much because our distributors wouldn't carry them anymore lol.

As for the rpi 5 combination, the power draw is relatively low. The whole thing clocks in at about 14 watts on an rpi 5 which allows us to run this platform off of a battery. With 26 tops, this setup can contend with the jetson xavier nx (21 tops, ~$500) and the jetson orin nano (40 tops, ~$500) for for a cost of around $170. Furthermore the cpu on the rpi 5 is generally more performant than the xavier nx.

Specifically, this is an excellent vision module for real-time object detection from multiple cameras if setup properly, while maintaining access to the prolific raspberry pi hardware ecosystem which is typically cheaper than the jetson ecosystem.