weidi_xie

nice

Indeed, the only difference is they work on RGB space, and the dataset is a bit toy-ish (no offence), as the networks simply need to separate the objects either by color, or a regular texture pattern.

What proposed in this motion grouping paper, is more like on the idea level, which gives an observation that, although objects in natural videos or images are of very complicated texture, and there is no reason a network can group these pixels together if no supervision is provided.

However, in motion space, pixels moving together form an homogeneous field, and luckily, from psychology, we know that any parts of the objects tend to move together.

The time for computing optical flow is indeed a bottleneck, and some researchers in the community is working on that: - https://arxiv.org/abs/2103.04524 - https://arxiv.org/pdf/2103.17271.pdf

If you check the Table 3, you can see the comparison with some of the top unsupervised video segmentation model, trained with supervised learning, eg. COSNet, MATNet. They perform reasonably well on MoCA, but they were all trained with massive manual segmentation annotations, which is not typically not scalable.

The proposed self-supervised approach is comparable to those top methods, even without using RGB, and any manual annotations.

will be online soon.