mcroydon
mcroydon on Hacker News
I highly recommend "Proving Ground: The Untold Story of the Six Women Who Programmed the World's First Modern Computer" for more context about how the ENIAC came together. It includes the point of view of the programmers of the ENIAC who are often left out of or diminished in other accounts.
https://www.hachettebookgroup.com/titles/kathy-kleiman/provi...
I'm still a little confused why the story posted the other day fell off the front page so quickly, there was some good discussion and a link to a youtube meta-review that gave a pretty interesting view in to the last few days of the company:
https://news.ycombinator.com/item?id=15095861
Not to be self-serving, but I've always been fascinated by Therac-25. I ended up doing a deep dive a few months back and put together a short 5ish minute podcast episode about it:
http://tinycast.in/2015/01/27/therac-25/
I used this PDF as one of the primary resources and it was a fascinating read.
Good call. The dataset was large enough that it didn't feel silly to use something like MapReduce but the same thought has been in the back of my head the whole time.
Thanks for the tip, I didn't think to copy the data to ephemeral storage like that. That'll probably speed things up a lot.
I ended up splitting the data in to a relatively small number (~200) of ~30MB gzipped files in order to initially saturate the mappers and speed things up. If that's not necessary after moving to ephemeral storage that's fine by me!
With 8 instances and the number of files the input was split in to there were definitely both map and reduce tasks waiting for a runner. I don't know exactly how much but I'm pretty sure I'm paying a pretty heavy IO tax by using S3.
Another +1 on this fantastic book. It's extremely accessible and concise while covering quite a lot of ground on the subject.
If only there were some good non-GPL machine learning libraries for Python...
The biggest thing I got from PCI was a basic discussion of a lot of algorithms that I was able to research further on my own. I had no idea that I wanted to know more about K-means clustering until I read the book.
If that's all you get out of it great, but there's a lot of interesting stuff in there, covered in enough detail to get you started, which is more than enough to make me happy.