I think the haystack folks made the right decision, here. A rule of thumb is that it takes 3-5 years for a novel on-disk format to gel. There are always random bugs that rely on particular sequences of allocations/deallocations, races, etc. Using an off-the-shelf filesystem that allocates the blocks and then stays out of the application's way, like XFS, probably saved these guys at least one year of screwing around.
Haystack effectively IS a filesystem - a log structured filesystem.