1. perhaps `.reduceLeft(_ + _)` could be replaced with the use of a `sum` function or method (assuming one exists in scala?)
2. if the `topKeywords` collection returned a default value with a `.score` of 0 when queried with a key it doesnt contain, the headOption getOrElse null match null would not be necessary.
e.g. in python it might look something like this:
Keyword = namedtuple('Keyword ', ['score', ...etc...])
top_keywords = defaultdict(lambda : Keyword(score=0, ...etc...))
def sbs(words):
if words:
return (1.0/len(words)) * sum(top_keywords[w].score for w in words)
else:
return 0.0
(apologies for making superficial comments about the code. the algorithm itself certainly seems interesting)One minor nitpick that can be of help when dealing with tuples: A partial function ( {case xxx => yyy} ) is a Function1 so you can use it with map and filter. This way you can deconstruct tuples into names and avoid using _1, _2, etc. { case (name, value) => blah }
https://github.com/MojoJolo/textteaser/blob/master/src/main/... could be made more readable by giving names to the tuple elements.
Thanks for publishing this code. It yields impressive results.
you post non-idiomatic(?) scala in a comment to explain what you are doing, i think? not a criticism of textteaser (which was behind this excellent project https://news.ycombinator.com/item?id=6498625), but seems to raise questions about the language...