You might try to contact William Stein. He is a former academic (tenured professor at the University of Washington) that was the lead developer for SageMath, an open source product which he's tried to commercialize along vaguely similar lines. He's now working on a project called CoCalc, which I know less about.

Unfortunately, my impression is that he's had some bad luck, despite a fantastic product, and also despite being 100% an insider whom many research mathematicians personally know and deeply admire. Last I heard, he found us a difficult market to sell to.

Maybe his luck has turned around, I certainly hope so. I see him participate on HN every once in awhile. His email address is [email protected] (not private -- I merely saved you two seconds of googling), and perhaps he would be willing to share his advice.