Re:Show HN: Word2vec Algorithm in ~100sloc with NumPy
by extasia on Thursday June 01, 2023 @11:35AM
(#36152650)
Hi eesmith, thanks for your insight! I've applied a lot of the improvements you mentioned, the collections.Counter one didn't even cross my mind- very neat:)
The lines are processed independently as they are separate sentences. One line may be from one source, and the next from a completely different source. The problem with simply concatenating them is that the target words on the boundaries between sentences can end up with context words from unrelated sentences.
>I liked seeing the 'η' - I haven't yet used non-ASCII in my code!
Then you may enjoy this april fool's python issue I made https://github.com/python/cpython/issues/103172
Yeah, I like using utf-8 chars for mathematical symbols, it makes my brain hurt a little less when mapping between the two! I also like using ŷ for predictions in ML contexts as that's the canonical symbol used in the literature.
Could you explain why lines are processed independently?
Does it make a difference that some lines have fewer words than others?I would think that
using the sliding_window and flatting recipes at https://docs.python.org/3/library/itertools.html#itertools-r... would be more appropriate. Though using it directly would not handle the leading/trailing partial windows.I liked seeing the 'η' - I haven't yet used non-ASCII in my code!