Facebook open-sources LogDevice, a distributed storage for sequential data

Re:Facebook open-sources LogDevice, a distributed storage fo... by otterley on Friday September 14, 2018 @12:10PM (#17988375)

I didn't think we were speaking of millions of topics here; only millions of logs. You can certainly have logs numbering in the millions using a single topic. Mux/demux would have to happen at the producer/consumer side, of course.

[ Parent | on HN ]

Re:Facebook open-sources LogDevice, a distributed storage fo... by manigandham on Saturday September 15, 2018 @11:41AM (#17994687)

Do you mean log segments then? In that case I don't see what's special about it because that's just rolling files and all of these systems can handle millions that way.

As far as millions of topics, if you have to do it at a logical layer yourself, then you might as well use a system that supports it natively.

[ Parent | on HN ]

Re:Facebook open-sources LogDevice, a distributed storage fo... by rakoo on Saturday September 15, 2018 @12:39PM (#17994913)

The logs in LogDevice also have an independent lifecycle, which your solution doesn't allow.

[ Parent | on HN ]

Re:Facebook open-sources LogDevice, a distributed storage fo... by sh00s on Friday September 14, 2018 @06:55PM (#17991750)

A log in LogDevice is roughly equivalent to a Kafka partition.

[ Parent | on HN ]