News for hackers. Stuff that matters.
Re:Heavy Metal and Natural Language Processing by mkozakov on Sunday July 03, 2016 @08:09AM (#12025760)
There is something to be said about song structure. Most songs will have a chorus, which is repeated throughout the song. Choruses will generally have higher 'readability' because they are meant to be more memorable. If you don't pre-process your corpus by removing repeating paragraphs, then your comparisons to the brown corpus are not as valid, since brown doesn't use repetition.
Re:Heavy Metal and Natural Language Processing by jessemoeller on Sunday July 03, 2016 @11:28AM (#12026311)
I believe that the convention on darklyrics is to either elide or replace with "[chorus]" all repetitions of the chorus after the first. A few spot checks seem to confirm this, but I can't find an official policy on it.