There's also people like my mother who believe marijuana is akin to meth. If you smoke it once you'll end up homeless, under a bridge down by the river
Another example to point to from recent US political history: the federal legalization of same-sex marriage, and subsequent rapid rehabilitation of the public platforms held by its former opponents still regularly running for re-election.
New York has all the political will needed but the problem is the bureaucracy. This should not be the least bit surprising.
Changes to law are coming from the courts, ballot initiatives, executive orders, and the bureaucracy, not from legislative bodies like they're supposed to. Legislators 'dunk' on Twitter all day and very little legislating actually happens.
It's an easy out for them -- they rarely go on record for supporting or opposing things now -- and can blame presidents, bureaucrats, and the courts for anything that turns out to be unpopular.
I'm guessing maybe it's because although those who oppose some form of marijuana (neither medical nor recreational) are in the minority, they still make up a sizable voting block when an election is won on the marginal vote. But how does that play out when >2/3 of people support it across the board, and that politician is the only one doing something to push it forward? That politician would literally be the only choice if marijuana is meaningful enough to a voter. So while the politician may lose some supporters, would they not gain support from others?
So they put it up for referendum instead, so it happens but it doesn't harm anyone's careers.
It's not at all any kind of vindication of direct democracy. Rather, because it's broadly popular but also highly unpopular with significant segments of voters on both sides... a referendum works out for everyone.