Re:Scribe: Transporting petabytes per hour via a distributed...
by johndoe345 on Monday October 07, 2019 @02:15PM
(#21184154)
This is the same scribe.
Facebook closed-sourced it because it was too hard to maintain an open version and a version that addresses Facebook's needs.
Re:Scribe: Transporting petabytes per hour via a distributed...
by naringas on Monday October 07, 2019 @05:41PM
(#21186242)
I wonder why couldn't they just make the open source version address their needs...
Re:Scribe: Transporting petabytes per hour via a distributed...
by javiermaestro on Monday October 07, 2019 @07:49PM
(#21187240)
The version that was open-sourced kept evolving and integrating with other internal systems at Facebook. That's what made it hard to continue open-sourcing it (why the open-sourced version was discontinued) and why the current version is also hard to open-source.
Maybe one day we'll have a version available. In any case, one of the larger parts of the system (LogDevice) is open source :)
(disclaimer: I work in Scribe)
LogDevice: https://engineering.fb.com/core-data/open-sourcing-logdevice...
Re:Scribe: Transporting petabytes per hour via a distributed...
by saisundar on Monday October 07, 2019 @04:44PM
(#21185760)
>"The growing number of complex components made it difficult to retain an open source version stripped of our internal specifications. This complexity is the main reason that we archived the open source Scribe project. The current version of Scribe’s architecture is detailed below, with a focus on the components that comprise the data plane. ". Quoted from the article.
Re:Scribe: Transporting petabytes per hour via a distributed...
by gnud on Monday October 07, 2019 @04:19PM
(#21185508)
Scribe is also an enterprise integration/ETL tool. For _many_ years before FB existed.
Re:Scribe: Transporting petabytes per hour via a distributed...
by javiermaestro on Monday October 07, 2019 @07:55PM
(#21187275)
Interesting! Just found out about this, and I googled it out. TIBCO acquired Scribe Software, and Scribe Software was running as early as 1998-1999! :D
https://web.archive.org/web/20120301000000*/http://www.scrib...
The initial stuff seems to not be so related, but the current description of what they do seems much much closer to what Facebook's Scribe does today :)
Naming is hard! :D
We used it at a company I worked for, but it had long-since been deprecated, so I was confused when I saw this Scribe.