I expect you'd have a pretty complicated jurisdictional case if a team of FBI agents were to go into a dispensary in a state where it was legal and round everyone up. And pretty much no sane politician would go anywhere near it. I also wouldn't (hypothetically) travel with the stuff over state boundaries.