You don’t think it’s more likely that a person who talked about fishing also searched for fishing gear using Google? Or “Likes” fishing on Facebook? Or follows a fishing person on Twitter?
The technology and storage that would be required to parse non-device-directed speech doesn’t exist and wouldn’t be profitable since there are so many other reliable signals that are much cheaper.
Voice Assist is yet another privacy invasion vector imho, there are too many anecdotal first hand accounts of someone talking about fishing and suddenly getting banner adds for boat trips everywhere.