Re:New High I/O EC2 Instance Type - hi1.4xlarge - 2 TB of SS...
by jedberg on Thursday July 19, 2012 @03:46AM
(#4264934)
It's not just random I/O that's fast, but sequential is a lot faster too.
But what I'm thinking of is the tokenization of a large corpus of text into bigrams and trigrams for example.
Re:New High I/O EC2 Instance Type - hi1.4xlarge - 2 TB of SS...
by cperciva on Thursday July 19, 2012 @04:22AM
(#4265022)
Good point, with all the focus on random I/O in the announcement I was forgetting about the sequential I/O performance benefit.
Hmm, that reminds me, I have an interesting toy problem which is sequential I/O limited...
Re:New High I/O EC2 Instance Type - hi1.4xlarge - 2 TB of SS...
by bravura on Thursday July 19, 2012 @03:53AM
(#4264953)
Admittedly, it would be fun to pull in the Common Crawl and process it.
Note that there's a code contest for Common Crawl that recently was announced: http://commoncrawl.org/first-ever-code-contest/
(Not trying to be argumentative, I'm just trying to figure out what you have in mind.)