Re:MongoDB has filed to go public
by gaius on Friday September 22, 2017 @11:36AM
(#15313109)
If you process this kind of high-throughput data in R then yep, you get it as 3 data.frames as you describe, it's nicely tabular and easy to "join" by just matching index numbers.
- FEATURES table (~400k rows), primary key: Feature_ID
- SAMPLES table (tens of thousands), primary key: Sample_ID
- OCCURRENCE table (what you observed), with the fields Feature_ID, Sample_ID, Occurrence_Count.
This is pretty much a standard solution well proven in data warehousing...