Understanding Google’s File System (2020)

Re:Understanding Google’s File System (2020) by kirugan on Thursday September 01, 2022 @06:54AM (#32674626)

Original whitepaper is a great overview of how the system works, but devil in the details. I still have a lot of questions about implementation, for instance:

from where did master get information about chunkservers (chubby, configuration files) ? did chunkserver know about which master it belongs to? etc.

[ Parent | on HN ]

Re:Understanding Google’s File System (2020) by lokar on Thursday September 01, 2022 @11:35AM (#32677906)

GFS predated chubby.

The master is configured with the endpoints for all the chunkaervers. A separate control/config system supported this including things like automatic replacement of dead nodes.

[ Parent | on HN ]

Re:Understanding Google’s File System (2020) by kirugan on Thursday September 01, 2022 @03:57PM (#32682209)

how do you know?

[ Parent | on HN ]

Re:Understanding Google’s File System (2020) by lokar on Thursday September 01, 2022 @08:11PM (#32685064)

I was there

[ Parent | on HN ]

Re:Understanding Google’s File System (2020) by kirugan on Friday September 02, 2022 @01:07PM (#32693164)

Awesome! Just checked the whitepaper there was a cluster with 300 machines, so all machines had to be described in configuration file that was used by master, right?

[ Parent | on HN ]

Re:Understanding Google’s File System (2020) by lokar on Saturday September 03, 2022 @11:00AM (#32702490)

Mostly. Binaries of that era took everything on the command line. The config system would generate that from files and keep the running binaries up to date via ssh.

[ Parent | on HN ]