News for hackers. Stuff that matters.
Re:Top Books on Amazon Based on Links in Hacker News Comments by rahimnathwani on Monday January 18, 2016 @08:05PM (#10928211)
This is really cool. I started reading HPMoR after finding it.

I'm curious: after performing NET on the corpus, how did you filter to find books only? Did you just search on Amazon's catalogue for exact matches, or was more tweaking required?

Re:Top Books on Amazon Based on Links in Hacker News Comments by _lpa_ on Tuesday January 19, 2016 @04:27AM (#10929545)
I built a little tool to help with that, but pretty much the first result on amazon was always right - for books with more than one match, at least.
Re:Top Books on Amazon Based on Links in Hacker News Comments by rahimnathwani on Tuesday January 19, 2016 @07:15AM (#10930190)
Thanks. How did you deal with, e.g. 'Steve Jobs'? I guess a generic NER tool would extract that as a single entity. Then you'd search Amazon and find an exact title match (the biography by Isaacson). But you probably don't want this in your list, as the references were to Steve Jobs the person, not Steve Jobs the book.

So did you use some manual review step to remove these?

Re:Top Books on Amazon Based on Links in Hacker News Comments by _lpa_ on Tuesday January 19, 2016 @07:38AM (#10930284)
Well, the nature of NER is that it can extract 'Steve Jobs' as either a PERSON or a BOOK entity (in my case). You can in principal extract whatever kind of entity you want, provided you have appropriate training data. The idea (loosely) is that it learns the structure of text around, say, a book. You can then label a piece of text as a book, despite not previously knowing that title. It doesn't solve the problem of two books having identical titles, however :).

(Note: I'm not an expert in NER/NLP, so please correct me if I have got something wrong!)

Re:Top Books on Amazon Based on Links in Hacker News Comments by rahimnathwani on Tuesday January 19, 2016 @10:39PM (#10935916)
Aha, I see. Thanks. I thought you were using some generic pre-trained NER, but I understand now that you used either one which was pre-trained to find books, or you trained your own.