Perhaps paradoxically it is easier to maintain a gigantic Python code base than it is to maintain one that is small or medium sized.

The reasoning, a gigantic code base means infrastructure and tooling investment will obviously pay off. It probably even means you get one or more dedicated resources of this sort. Whereas a small to medium sized codebase might bump into some of the same issues as a larger code base but not justify such investments.

If anyone else is looking: https://www.youtube.com/watch?v=ZYD9yyMh9Hk :) I'm definitely interested in this - I've seen a struggle to manage a far smaller codebase than that in Python and really interested to see what you're doing differently to make it work.

The most concise answer is 'tooling'. We have a DevTools team that builds the SDLC and automated testing infra, all tightly integrated into MS Visual Studio. That lets the thousands of developers focus on productive coding.

Just watched it. Thanks, it was indeed a great talk, and indeed a lot of tooling.

Do you have any extra writeup to suggest?

There isn't a ton of detailed public information on this kind of stuff. My talk focused on the general principles of our approach, rather than the myriad of features and optimizations necessary at such a large scale.

A good next step would be reading the 2016 ACM paper "Why Google Stores Billions of Lines of Code in a Single Repository" [1] then reading a recent HN discussion [2]. That has a ton of good links.

[1] https://cacm.acm.org/magazines/2016/7/204032-why-google-stor...

[2] https://news.ycombinator.com/item?id=17605371

Is this made as a cluster of independent programs or is this actually a monolith, where you can access all functionality from some python method or class?

To grossly oversimply:

- Yes, it is a monolithic application architecture. There are distinct frameworks (e.g. GUIs, DBs, distributed compute, pubsub, lazily evaluated graph, reactive graphs, etc) designed to be used together in a unified way based on a global object store.

- A 150 person platform dev team writes these core frameworks and dev tools and runs the SDLC infra.

- Then the several thousand developers in business teams build applications for their particular needs.

- When the platform team sees businesses building their own solutions to a common need, it will productionize that as a new core framework for all business to use.

- In Python terms, there is a relative small number of core classes, and very many application-specific subclasses.

I used to work on the subsequent brain child of Kirat Singh et al at another big financial institution, and ditto. Admittedly we used a custom built source control system and release tooling, but also developed in Python.

Is there a quick way to find out your LOC in a python project? Does it include whitespace and comments? What about HTML and JS code?

The typical measure is “Source Lines of Code” (SLOC), which would be calculated on a per-language basis and doesn’t include comments or blank lines. You’d sum these together for your codebase overall. Not sure how you’d do it for HTML or mixed code. Presumably the same since it’s such a rough measure that that level of accuracy really doesn’t matter.

I'm using tokei for that. EDIT: it gives you the number of lines of code, blank lines and comment lines, for each language in your project.

cloc is another commonly used package. It handles multiple languages and has a nice, readable output format.