I've had this on my mind in various forms for almost a decade. It's only recently been crystallized from something abstract into much more concrete thanks largely in part to Bret Victor's talks. If you haven't already seen them, I highly recommend "Stop Drawing Dead Fish"[0] and "The Future of Programming"[1] as starting videos. Then just watch all the rest of his talks. :)

[0] https://vimeo.com/64895205

[1] https://vimeo.com/71278954