News for hackers. Stuff that matters.
Re:Algorithms Behind Modern Storage Systems by akeck on Tuesday May 15, 2018 @10:03PM (#17079447)
This reminds me of RAID 6’s use of abstract algebra.
Re:Algorithms Behind Modern Storage Systems by akalin on Wednesday May 16, 2018 @12:57AM (#17080102)
I wrote a blog post a while back that might be of interest: https://www.akalin.com/intro-erasure-codes . It's similar to Igor's but fills in a few more details.
Re:Algorithms Behind Modern Storage Systems by jorangreef on Wednesday May 16, 2018 @04:02AM (#17080732)
That's a fantastic introduction (although it's far more than an introduction!). I wrote a native addon for Node.js that does Reed Solomon, also using optimized Cauchy matrices (MIT license): https://github.com/ronomon/reed-solomon
Re:Algorithms Behind Modern Storage Systems by akalin on Wednesday May 16, 2018 @04:18PM (#17086086)
Very nice! I think I stumbled on your implementation when researching my post. :)
Re:Algorithms Behind Modern Storage Systems by jorangreef on Thursday May 17, 2018 @06:25AM (#17090125)
Thanks. If that was in November last year, then it's changed a lot since then (and now a few times faster). Back then it was using a Vandermonde matrix as it was based on Backblaze's Java implementation, but a month ago everything was rewritten to use optimized Cauchy matrices.
Re:Algorithms Behind Modern Storage Systems by gameshot911 on Tuesday May 15, 2018 @10:44PM (#17079645)
Link below with more info, for those who were interested like me!

http://igoro.com/archive/how-raid-6-dual-parity-calculation-...

Re:Algorithms Behind Modern Storage Systems by hinkley on Wednesday May 16, 2018 @12:06AM (#17079941)
Reed Solomon Coding.

Linear algebra. It’s so hot right now.

Re:Algorithms Behind Modern Storage Systems by vanderZwan on Wednesday May 16, 2018 @10:12AM (#17082511)
Until recently I worked for a molecular neurobiology research group that, among other things, used Single-molecule RNA Fluorescence in situ Hybridization (smFISH) to measure gene expression in tissue[0].

Basically: design viruses with fluorescent molecules attached to them such that they attach to specific sections of RNA in the cell that are associated with particular genes. Soak tissue in viruses. Look at tissue through a microscope and count individual fluorescent dots, each of which represents one RNA molecule (I find this absolutely mind-blowing). Wash off the viruses with a specific type of chemical, and repeat with a new one for a different gene.

You can only use a few colours at a time because otherwise the microscope cannot discern them. But that would severely limit the throughput - there are a lot of genes we want to check, but the tissue will also degenerate after repeated washing. So, what can we do? Well, as I've understood, scientists using smFISH and similar techniques now use multiplexing and with Hamming codes to get around that.

So yeah, linear algebra is definitely so, so hot right now.

[0] http://linnarssonlab.org/osmFISH/

Re:Algorithms Behind Modern Storage Systems by bogomipz on Wednesday May 16, 2018 @06:46PM (#17087076)
Doesn't RAID 6 just use Reed-Soloing/Hamming codes? Is that based on abstract algebra?