> This code does a little of that.

Actually, the code does none of that ;-) All of the semantics are provided by the users: either as manual annotations or by plugging in an existing architecture for semantic segmentation / pixel labeling. It's designed to be independent of the source of the semantic maps, so we can continue to work on both problems separately.

It works for basic color segmentation already, and here are some of the papers we're integrating currently: http://gitxiv.com/search/?q=segmentation