News for hackers. Stuff that matters.
Re:Ask HN: What are the best tools for analyzing large bodie... by abannin on Thursday June 18, 2015 @02:30AM (#9737103)
I'd like to understand a bit more about your data structures. Raw text should live in a file system, but if you're creating highly structured data using a SQL database could be a good fit. I'd also suggest taking a look at Spark; you can run it locally on your machine (does not need a Hadoop cluster) and can interact with flat files. If you're finding that your machine is lacking in horsepower or space, AWS could be your friend.

There are no comments on this comment yet.