You can use tr instead of sed and grep, too. Also -k1 and -n 10 are default for sort and head:
$ < bible-pg10.txt tr -cs a-zA-Z '\n' | sort | uniq -c | sort -nr | head
62265 the
38915 and
34588 of
13474 to
12846 And
12589 that
12387 in
9762 shall
9668 he
8942 unto
I have a `bins` script that is just `sort | uniq -c | sort -nr`, so that reduces to `tr -cs a-zA-Z '\n' | bins | head`.Unix for Poets, man. It's the shit.
I ALWAYS forget about squeeze, thanks!
Re:LINQ Ruined My Favorite Interview Question
by stass on Thursday July 11, 2013 @05:18PM
(#6029295)
This is too complicated...
echo $sentence | rs -T | sort | uniq -c | sort -rn|head -10
Ooh that's a clever use of reshape, but wrong sadly. Yet you're right I could make it simpler, it was just what came first to mind, I did not even check it. Imagine typical sentences:
$ echo Foo foo foo. | rs -T
Foo
foo
foo.
$ echo Foo foo foo. | tr 'a-z' 'A-Z' | sed 's/[^A-Z]/\
/g' | grep -v '^$'
FOO
FOO
FOO
Also the link mentioned using wiki articles for testing, so they would have paragraphs and that's where reshape shines for things like emails, but fails here: $ cat foo
Foo
foo
foo.
$ <foo rs -T
Foo foo foo.
$ <foo tr 'a-z' 'A-Z' | sed 's/[^A-Z]/\
/g' | grep -v '^$'
FOO
FOO
FOO
That that's though makes me think, should I treat contractions special? What about plurals? It's starting to get silly now.
Re:LINQ Ruined My Favorite Interview Question
by stass on Thursday July 11, 2013 @08:14PM
(#6030120)
I think it goes beyond the original question now. :) If one wants to do proper thing, then definitely contractions, word variations, etc should be considered. For this, I guess something like OpenNLP can be used.
What's interesting though is how UNIX shell, being essentially a symbolic FP language, allows one to solve the problem in a clear and concise way. And if one desires, the program can be easily modified to read sentences, say, from network from ssh tunneled via HTTPS. That kind of flexibility can rarely be achieved in languages like C# with tight coupling between the units of abstraction.
tr 'a-z' 'A-Z' | sed 's/[^A-Z][^A-Z]*/\ /g' | grep -v '^$' | sort | uniq -c | sort -nrk1 | head -10
edit: I don't know enough about HN,there should be a newline after the backslash in the sed command.