It is indeed illegal on the federal level. Currently everyone is just an executive order away from crack down on all of it. The guidance that the Obama Administration has been using is not to enforce the law in states where it has been legalized (though they have been cracking down on cases where dispensaries open within 1000 ft of a school). That said, a new president could do whatever they wanted and a crackdown would likely result in an eventual supreme court case that the states would probably lose based on current precedent.