FWIW, Google doesn't have a JVM bias in terms of its big data stack, but it still has langauge biases. Bioinformatics is built on Unix tools and Unix composition AFAICT, and that philosophy needs to make its way back to the big data stack IMO.
My question is what's wrong with what they're doing? If it's working, why not keep doing the same thing, rather than trying to move to Hadoop, which sounds like the wrong tool?
I agree.
> Bioinformatics is built on Unix tools and Unix composition AFAICT
That is true for most bioinformatics tools with a few important exceptions. CummeRbund, e.g., is a must for RNA-Seq and you need to drive it from R. It's hard to tell whether supporting only command line would actually be enough to conveniently describe workflows. For Cuneiform, we assume that it isn't.
> what's wrong with what they're doing?
It's not that moving to Big Data stacks is the ultimate answer to everything. If your workflow runs fast enough on one machine, probably it's best to change nothing. Nevertheless, we're observing a dramatic drop in sequencing costs right now. And the more this trend continues, the bigger will be the demand for scalable infrastructures in Next Generation Sequencing.
- write a wrapper (an approach taken by Crossbow [1], SeqPig [2], or BioPig [3])
- reimplement (an approach taken by CloudBurst [4] or ADAM [5])
Reimplementing a whole pipeline represents a huge upfront cost. Writing wrappers is more feasible and, accordingly, a huge amount of custom tailored pipelines and ad-hoc worklfow systems have emerged recently. So if it boils down to wrapping libraries and running them on Hadoop, then we need to make this as easy as somehow possible. Cuneiform is a step in this direction.
[1] http://bowtie-bio.sourceforge.net/crossbow/index.shtml [2] http://bioinformatics.oxfordjournals.org/content/30/1/119.lo... [3] http://bioinformatics.oxfordjournals.org/content/early/2013/... [4] http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2682523/ [5] http://bdgenomics.org/projects/adam/