I think Chomsky follows the definition of socialism as you, "collective ownership of the means of production," but where he differs is that his definition doesn't include "government ownership of the means of production." To anarchists, like Chomsky, if any hierarchical structure controls the means of production, it is not truly collectively-owned. He and other anarchists use the term "state capitalism" to describe the USSR, since it's capitalism where one hierarchy (the state) controls capital, instead of private entities as in liberal capitalism.

I haven't seen the video yet, but you don't quote him as saying that the USSR wasn't Marxist, just that it wasn't socialist. A huge historical difference between Marxists and socialists is how to transition from capitalism to socialism. Marx believed that a "worker's state/council" would be needed, whereas the anarchists predicted that a "red bureaucracy" would only be worse than the one it replaced.