I wonder if someone has already tried to do that. Though this might go in a similar direction: https://arxiv.org/abs/1711.00043