News for hackers. Stuff that matters.
Re:New High I/O EC2 Instance Type - hi1.4xlarge - 2 TB of SS... by jedberg on Thursday July 19, 2012 @03:46AM (#4264934)
It's not just random I/O that's fast, but sequential is a lot faster too.

But what I'm thinking of is the tokenization of a large corpus of text into bigrams and trigrams for example.

Re:New High I/O EC2 Instance Type - hi1.4xlarge - 2 TB of SS... by cperciva on Thursday July 19, 2012 @04:22AM (#4265022)
Good point, with all the focus on random I/O in the announcement I was forgetting about the sequential I/O performance benefit.

Hmm, that reminds me, I have an interesting toy problem which is sequential I/O limited...

Re:New High I/O EC2 Instance Type - hi1.4xlarge - 2 TB of SS... by bravura on Thursday July 19, 2012 @03:53AM (#4264953)
Admittedly, it would be fun to pull in the Common Crawl and process it.

Note that there's a code contest for Common Crawl that recently was announced: http://commoncrawl.org/first-ever-code-contest/