You're doing something wrong. I've used awk to run big data reformatting jobs in under an hour that took most of a day to run in Scala on an Apache Spark cluster. In the vast majority of cases today, if speed is your problem, then you are the problem - especially since most problems fit into RAM these days, even w/o exotic stuff like RAMcloud...