Re:Command-line tools can be faster than your Hadoop cluster
by kevin_thibedeau on Sunday January 18, 2015 @08:26PM
(#8909802)
In a real production environment that command line would be put into a script parametrized with named variables and the embedded awk scripts would be changed to here-docs.
[ Parent
| on HN ]
Re:Command-line tools can be faster than your Hadoop cluster
by mc808 on Sunday January 18, 2015 @09:04PM
(#8909909)
Sounds good although at that point it's just programming, and there are tools that are cleaner and faster and more robust than piping semi-structured strings around from a command line.
The one real benefit that can be argued is ubiquity (on *ix). Not every system has Perl, Python, or Ruby installed - or Hadoop for that matter - but there's usually a programmable shell and some variant of the standard utilities that will get something done in a pinch. If it happens to be 200x faster than some enormous framework, so much the better.