Re:Multi-Armed Bandits, Conjugate Models and Bayesian Reinfo...
by achompas on Saturday September 01, 2018 @11:02AM
(#17892141)
Does the fact that sklearn reverses signs for features that collide change your opinion?
Re:Multi-Armed Bandits, Conjugate Models and Bayesian Reinfo...
by lsorber on Saturday September 01, 2018 @01:48AM
(#17890459)
How are you compressing the feature space in that case, by truncating the trie?
Re:Multi-Armed Bandits, Conjugate Models and Bayesian Reinfo...
by ma2rten on Saturday September 01, 2018 @02:16AM
(#17890535)
You could use clustering, dimensionality reduce or feature selection.
However, the way I have seen the hashing trick being used is not to compress the feature space. For most problems it would be a bad idea to just lump your most discriminative features together with some other random ones. Instead people just choose a very large feature space which makes collisions unlikely. For model implementations using sparse matrices it doesn't matter if the feature space is very large. The main advantage of this is that you don't have to keep an expensive hash map of your vocabulary in memory (hence my suggestion to use a trie).
I'm not a huge fan of that. Hash collisions can lead to unexpected behaviors in production and make feature attribution for debugging harder.
It's slightly more effort to implement, but with a trie data structure you can store even the biggest feature mapping in memory.