It was probably fasta of fastq format, which is pretty much line separated strings of the DNA letters. BSON won't help with that. You could try to squeeze the each of the 4 letters into 2 bits, but they use few more letters to indicate "unknown", so it's not that easy. And even the simplest compression algorithms just do that for you.

And text is easier to work with in any hacked up script.

There are some binary formats (BAM, I think), but people often prefer the text format anyway. When compressed, the size is pretty much the same