I think you can read about some of these changes in Google's SRE and SWE books (even if they don't mention this video in particular), at least the ones most likely to be interesting to someone outside Google.

But dropping Borgmon readability was the most immediate and obvious. It was basically true that no one had Borgmon readability. The policy was a catch-22: you couldn't get readability for the simple/formulaic Borgmon macro invocations that were encouraged and often sufficient. You could only get it for doing something "clever". I got it by writing fancy borgmon rules to paper over a problem that (in hindsight) I should have solved elsewhere.

Another was easing quota management. IMHO the most unbelievable thing in the video was that after Broccoli Man told Panda Woman to get quota in two cells, she just said "done". Besides the hassle in transcribing what you needed into the request system [1], various types of quota were chronically unavailable where you needed them, even in tiny amounts. In 2010, I kept a critical infrastructure service running by regularly IMing major clients' on-calls asking them to donate 0.1 cpu(!) of their quota in some cell or another when I didn't have quite enough to grow. There was a "gray market" mailing list where people would trade resources they couldn't get through the primary system. But eventually, they built a system that for small services would make the quota just happen for you.

Overall, it was a kick in the pants for the most basic infrastructure teams that made them see how unnecessarily hard this is for their internal customers, prompting them to make small things just happen while keeping large things possible. In any large organization, it's healthy to get this kind of feedback regularly. The actual specific changes and technologies are pretty specific to Google in 2010...

[1] Many people managed this very very tediously with spreadsheets. I eventually wrote a tool to generate the requests based on comparing your intended production config with your current quota.

When people wonder why a large corporation would pay to be on AWS/GCP/Azure rather than running their own infrastructure this is one reason why.

Every single buzzword uttered in the video has a cloud equivalent except for PCR. These days people just give up on regional (even zonal) outages and just take downtime and blame their cloud provider in the rca so doesn’t matter. The big thing that i think is missing in this thread is just how enormous google infra was even at 2010 so these problems were and still are sort of unique

Would it violate Godel's incompleteness to try and have Google implement its resource provisioning in GCP?

I think I may have used your tool at some point. There were times in geo where we'd run out of bigtable quota and I needed to hit the "emergency loan" button to keep maps from going down globally.

That's why you had these problems: you should have just let maps go down globally and then pointed the finger at the appropriate target to blame for it.

I'm only half joking about that too. The other half is that sometimes it is better to let the system collapse under its own weight when that's what is required to convince the right people the system is broken.

Production priority quota horse trading in the days before it was easy was a real skill. But non-production quota was free and virtually infinite, even in those days.

Or converting between priority levels or users that were under your product. If you ran a large enough service, there was also begging your users for spare quota or warning them that their recent donation couldn't be fully deployed because a necessary change in some secondary borgcfg template had eaten into the service's resources, so it would help everybody if they knew anyone with a few cores laying around...

By the way, do you happen to be the jwb that filed a... creative BT quota request?

Yeah that's me.

Hah, I was the one that played along and made you keep going ("May you be gifted with a long, healthy life and an exquisite taste in ties"?). And I think SAD was one of the teams that occasionally helped us find resources in this or that cluster, along with Analytics and, later, Social.

AAAAA+ GREAT CUSTOMER WOULD SELL TO AGAIN

I seem to remember hearing prod quota disputes described as “monkey knife fights” by SRE. SWE would joke about using corp credit cards on EC2 instances rather than waiting on the outcome.

wow, that’s incredible. Doesn’t sound like real life.

Thing is, though, something equally ridiculous happens at every large company and at more than a few small companies. Worse in government. The important thing is how they react when it's pointed out.

The most obvious change that came from this video is that Google abandoned the Borgmon readability requirement. At Google, every change needs approval from someone who has passed a detailed style-guide review process in the given language.

Now over multiple changes, it used to require one fairly big one. It's still a pain in the languages that require it--which is all the main ones, but very few of the niche ones.

Many other things changed as well. Much of what the video complains about got automated and better documented. But the company has grown so much, and the product lines have diversified so dramatically, that there are still plenty of places to complain about the overhead.

Context on borgmon: https://sre.google/sre-book/practical-alerting/

borgmon was a truly weird system.

It still is, but it used to be, too.

an unexpected mitch hedberg line appeared from the tall grass!

I've proudly managed to avoid Borgmon in favor of Monarch. Which was new at the time, but worked all right even back then. I have a lot fewer gray hairs because of that. They should have kept and rigidly enforced the Borgmon readability requirement to force people to migrate off that convoluted, idiosyncratic piece of shit.

I've never understood places with rigid style guides policed by people. Its idiotic, because we have computers and in places like google presumably a fair number of them know basic parsing/lex sufficiently that if they can't make a tool like clang-format that automatically reformats on save/commit/whatever then they can use a tool like clang-tidy to toss warnings during a development/CI/whatever phase.

Putting people in charge of formatting/style is just an excuse for wasting time bikeshedding, either the code is wrong and a tool can tell you, or its not wrong.

Readability is not about formatting, that's an orthogonal issue. It's possible to have terrible code that's perfectly formatted.

It's more about good usage of idiomatic language constructs, which still requires good human judgement to evaluate.

Then why do you often need external readability reviewers, especially if you or your team don't contribute to a codebase frequently? Having humans judge other humans' style is a given; it is part of the PR process. But why do you _also_ need someone with a readability rating as opposed to a code owner?

Note: I'm not a Googler nor a Xoogler, but my partner worked at Google for some years and so I've heard her complain a lot about Google and readability.

> But why do you _also_ need someone with a readability rating as opposed to a code owner

FWIW, these can be the same person - if the have appropriate readabilities. The problem is that it's not easy to get readability in the first place, so occasionally teams have to look for people outside.

Right but the fact that these are two separate roles strikes me both as bureaucratic overhead and a fundamental lack of trust in individual engineers. If I were approached to enact something like readability, I would veto it immediately. We hire engineers because we trust them. Part of that trust means that engineers occasionally wade into codebases they don't understand in languages they don't know and need to make contributions, but we trust them to not make a mess. Do some of them make messes? Undoubtedly. Is the magnitude of the mess, multiplied by the number of engineers, worth the creation of an entire readability certification process? I don't think so.

> Is the magnitude of the mess, multiplied by the number of engineers, worth the creation of an entire readability certification process? I don't think so.

And that's an acceptable trade-off for a different sized company. Google arguably has the biggest centralised codebase in the world, and simply has different requirements.

As other have said, most of the overhead is from attaining readability, not the requirement itself.

> And that's an acceptable trade-off for a different sized company. Google arguably has the biggest centralised codebase in the world, and simply has different requirements.

AWS does not operate this way nor do many CDNs operate this way nor do ISPs operate this way. There's other high scale businesses out there. Google isn't the only one. Using Google's scale to justify business practices is a self-fulfilling prophecy.

> AWS does not operate this way nor do many CDNs operate this way nor do ISPs operate this way.

I'm not denying that, but all the same, we're not talking about thousands of separate codebases. It's a single monorepo codebase with consistent styling, the advantage being that people can switch teams or contribute to other Google software under the same style guide.

The overhead isn't a bug, it's a core feature as consistent style makes it that much easier to switch teams.

It's also worth noting that Borgmon readability included an awful lot of "how not to shoot yourself in the foot in strange and unexpected ways". That required someone who had shown they knew of those strange and unexpected ways to review your code.

And I take it google has done wide ranging scientific studies about the variations in coding styles and language constructs that it is a secret advantage that they know how to write "readable" code? Implying they tried a bunch of diffrent ways until settling on the one true way that allows a diverse set of people with diverse experiences to read it?

Ever heard of COBOL?

Because readability has always been in the eye of the beholder, and codifying it makes it even worse.

Oh boy. So, to start, we have done scientific studies about the costs and benefits of readability (as codified circa 2018, since that was when the last study was).

It was studied for multiple programming languages Google uses.

It was done, at my request, by the engineering productivity research team, who are experts in this kind of research - you can find public papers they publish[1].

For background: I was the person responsible for production programming languages, and I did this precisely because i did not feel at the time there were recent good studies as to whether readability was really worth the cost.

The answer is "yes, it is".

There is an upfront cost, but readability has a meaningful and net positive effect on engineering velocity.

It is large enough to make the cost back quite quickly.

One could go down the rabbit hole of seeing whether you improve (or make worse) the numbers by changing various parts of the style guides, but like I said, what is there now, has in fact been studied scientifically.

Honestly, i'm not sure why you think it wouldn't be. You come off as a little immature when you sort of just assume people have no idea what they are doing and don't think these things through. You are talking about a company with 60k+ engineers. Every hour of all-engineer time you waste is equivalent to having ~30 SWE do nothing for a year.

Maybe there are companies happy to do that. I worked at IBM for a few years when i was much younger ;). All I can say is that as long as i'm involved in developer tools at Google, i'm gonna try not to waste my customers time.

[1] I say this in the hopes you don't go making assumptions about whether the studies were done properly. They were properly controlled for tenure, number of reviews, change size, etc.

I'd love to see the research here. Because searching the software engineering Google papers from 2021-2021, the only somewhat relevant papers I found were case studies (such as [1] and [2]), which were based on Googlers' perception of their own processes. While self-perception can indeed by used to measure efficacy, there's a reason why it's used with great caution in surveys. I'll be honest, I have a hard time believing in readability myself.

[1]: https://research.google/pubs/pub47025/

[2]: https://research.google/pubs/pub43835/

We have never published the readability research. I could look into doing so, but it is awfully Google specific. Trying to come up with something generally applicable would be a very different project

I'm not a fan of readability exactly the way Google does it, but I'm pretty happy that Google insists on various aspects of it, like good identifier names.

I don't know of any research off hand, but I'm pretty sure the industry consensus is that good identifier names improve the quality of the code (Go style notwithstanding.) Readability is one way to training engineers to do it.

> Because readability has always been in the eye of the beholder, and codifying it makes it even worse.

This is empirically false. Consistency, even if it is unfavorable to your preferences, is superior to inconsistency. So a codified set of best practices is better than none at all.

There are part of Google's style guides that I would change if I could, but I also prefer having a style guide (and one that goes beyond things that are lintable) than none at all, because consistency across the codebase means that I can usually understand code at a glance, or if not, know at a glance that something unusual is happening. (this is in fact precisely the argument in favor of autoformatters like gofmt/black/prettier, but extended to softer concepts that can't always be formatted: consistent style, even if it isn't your favorite, is superior to inconsistent style).

> Consistency, even if it is unfavorable to your preferences, is superior to inconsistency. So a codified set of best practices is better than none at all.

Indeed, but at what cost? Every step of added gatekeeping reduces development velocity, and Google does not have a monopoly on high scale services (cough AWS cough). In an ideal world for every full-time dev writing code, you'd have a full-time reviewer whose job it is to simply review that dev's code. But the real world has budgets and deadlines, and humans become discouraged when changes take too long to complete. So is readability worth the added gatekeeping cost is the question.

You're ignoring the cost of unrestrained development without standards. The impression I've gotten is that a AWS papers over their lack of development gatekeeping with unreasonable oncall loads. All things considered, the "cost" of readability in terms of development velocity is fairly low. If anyone on your team has readability [in the language], they can review your code. This can be problem if only one person has it, but usually once one person gets through the process, you can get more people through fairly quickly.

I'm a bit of a polyglot at Google (I have readability in C++, python, and Javascript/Typescript, and am part-way through the Go and Java processes, which covers basically every popular language at the company), and IMO, the readability process for each language has been a net positive, although it was painful for a while when I had no readability and none of my coworkers did (but there are procedures for getting around that today).

For example, I "donate" my readability, and review random changes from people who don't have readability but need a quick turnaround or aren't yet working to gain readability.

> You're ignoring the cost of unrestrained development without standards.

I'm not. I'm not saying that code should be merged without any stylistic commentary; I'm simply proposing that stylistic commentary be provided by other stakeholders in the service you're contributing to, on top of whatever linter or sanitizer is being used. That is, after all, the goal of a human reviewer. To add _another_ layer of gatekeeping is what I'm questioning.

> The impression I've gotten is that a AWS papers over their lack of development gatekeeping with unreasonable oncall loads.

I don't agree with this perception at all. AWS spends a lot of time building frameworks and libraries which attempt to minimize human gatekeeping to only the necessary areas. So think retries, backoff, timeouts, circuit-breakers, etc. Instead of adding gatekeeping at the code level (with things like code style), instead gatekeeping is added at the architecture and operations stages, which is where most of the work of keeping a production level service up, available, and fast. Operational level gatekeeping in the bounds of a given organization is a much more neatly constrained problem than stylistic gatekeeping and this gatekeeping happens on a per-service basis rather than a per-CL/review basis. This keeps overhead a lot lower.

My impression from being outside of Google (and having a Xoogler for a partner) is that Google loves bureaucracy and process, and that engineers that continue to stay at Google are fine with navigating these processes. Its answer to problems is to add more bureaucracy, another item for the checklist that a person/committee/reviewer needs to check for. That reflex is hard to shake, IMO.

> That is, after all, the goal of a human reviewer. To add _another_ layer of gatekeeping is what I'm questioning.

I guess I'm not following the distinction. For most people, that's exactly what happens. Either you, or the someone on your direct team, will be the person with readability for your code. The readability process just ensures that you don't end up with silos where style diverges too much, and that the stakeholder who proposes stylistic commentary has a baseline knowlege of linguistic best practices.

With few exceptions, the thing people complain about is attaining readability themselves, not getting changes approved.

> I don't agree with this perception at all. AWS spends a lot of time building frameworks and libraries which attempt to minimize human gatekeeping to only the necessary areas. So think retries, backoff, timeouts, circuit-breakers, etc.

I think we're talking a bit past each other. These things are all table stakes defined by frameworks and updated by centralized processes as best practices change. Human reviewers, for the most part, are only going to say "why are you diverging from the default".

But importantly, that sidesteps my statement entirely, which, rephrasing, is that amazon carries along more technical debt than Google. Whether that's a good long-term business decision remains to be seen, but the impression I get from ex-amazon friends is that maintainability is less prioritized, and maintainability is one of the goals of the readability process. Global consistency helps anyone understand and improve anything else, and to an extend avoids haunted graveyards (though...not entirely).

> instead gatekeeping is added at the architecture and operations stages, which is where most of the work of keeping a production level service up, available, and fast. Operational level gatekeeping in the bounds of a given organization is a much more neatly constrained problem

I'm not sure what you mean, but if anything, this sounds more bureaucratic.

> My impression from being outside of Google (and having a Xoogler for a partner) is that Google loves bureaucracy and process, and that engineers that continue to stay at Google are fine with navigating these processes. Its answer to problems is to add more bureaucracy, another item for the checklist that a person/committee/reviewer needs to check for. That reflex is hard to shake, IMO.

FWIW I don't get this impression. It's honestly difficult for me to tell what you even are referring to. Like, other than readability and promo, there's very few processes or bits of bureaucracy that are consistent across the company, and those two have gotten decidedly more streamlined since I joined the company.

Well okay that's half true, I can think of a bunch of places where bureaucracy has increased, but they're all spurned by legislation.

> I guess I'm not following the distinction. For most people, that's exactly what happens. Either you, or the someone on your direct team, will be the person with readability for your code. The readability process just ensures that you don't end up with silos where style diverges too much, and that the stakeholder who proposes stylistic commentary has a baseline knowlege of linguistic best practices.

Separating these two out, even if one person may fulfill two roles, IMO adds to bureaucracy that I don't care for. Ramping up onto a new language sounds like a pretty fraught experience if nobody on your team has readability and the service you're contributing doesn't have a member who has the time to actually work with you, especially if your work is not relevant to theirs. They even touched upon this in the linked video. The whole idea that you can enter a bureaucratic catch-22 at a company seems like an anti-pattern to me, a moment to stop and think; a "modern" engineering organization should try its utmost to accelerate development, with only as many checks as needed for safety and no more.

> With few exceptions, the thing people complain about is attaining readability themselves, not getting changes approved.

That's what I mean by bureaucracy. Even having "another eye" on the code should work. Having a certification process for readability is even more bureaucracy.

> But importantly, that sidesteps my statement entirely, which, rephrasing, is that amazon carries along more technical debt than Google. Whether that's a good long-term business decision remains to be seen, but the impression I get from ex-amazon friends is that maintainability is less prioritized, and maintainability is one of the goals of the readability process. Global consistency helps anyone understand and improve anything else, and to an extend avoids haunted graveyards (though...not entirely).

Ah, that makes a lot more sense. In my career I've never worked at a place that prioritizes code style that much, and I've worked at other high scale places in the past. This seems like a Google specific need. You're right that global consistency helps to save off "there-be-dragons" codebases, but I feel like this is a case of YAGNI. Unless you have engineers constantly cross-cutting across the company, most engineers learn the style guidelines of a team/product/service by ramping up on the team. The optimization for global consistency seems premature to me.

> FWIW I don't get this impression. It's honestly difficult for me to tell what you even are referring to. Like, other than readability and promo, there's very few processes or bits of bureaucracy that are consistent across the company, and those two have gotten decidedly more streamlined since I joined the company.

But promos drive _so much_ of the company culture, at least from what my partner tells me. And the video in the OP certainly makes it sound like Google has lots of bureaucracy, though I'm not sure how many of the requirements outlined in the video were there for hyperbole or not. But especially for services that have low SLOs, it makes no sense to put thought into failover, evacuation, or anything like that.

> Separating these two out, even if one person may fulfill two roles, IMO adds to bureaucracy that I don't care for. Ramping up onto a new language sounds like a pretty fraught experience if nobody on your team has readability and the service you're contributing doesn't have a member who has the time to actually work with you, especially if your work is not relevant to theirs. They even touched upon this in the linked video. The whole idea that you can enter a bureaucratic catch-22 at a company seems like an anti-pattern to me, a moment to stop and think; a "modern" engineering organization should try its utmost to accelerate development, with only as many checks as needed for safety and no more.

Keep in mind this video is ten years old. While yes, trying to write something from scratch in a language no one on your team has any experience in can be fraught (although this raises other questions: what exactly are you doing, in every case I've used a new lang, my team has been able to find people to review if needed), even in that case, the organization has help for you (see again my prior comment about "donating" readability, and my understanding is that the readability granting process actively prioritizes people who "need" the readability because they have few potential reviewers).

> That's what I mean by bureaucracy. Even having "another eye" on the code should work. Having a certification process for readability is even more bureaucracy.

I think initially the primary driver of readability is C++ (but Java + Guice also...needs it). Take a look at https://abseil.io/tips/. That's 70 tips, which is around a third of the internal ones. I have C++ readability and have internalized, some of those. The readability granting process ensures your code is reviewed by someone who knows all (or nearly all) of those tips, and will recognize when you're doing wrong things in your code and help explain how or why you can improve. When I personally went through the C++ readability process, I got mentorship on how to fix bugs in my code, some of which a peer on my team would have caught, and some of which they wouldn't have. I can now recognize those bug prone patterns (which are hard to lint for, in my case they mostly had to do with parallelism) and know about tools to debug them (msan and asan, which an experienced C++ user should know, but on my team of people who had basically no C++ experience at Google? Nah).

That follows into other languages. Being granted readability means you have a familiarity with the language that suggests you'll be able to effectively mentor others and not mislead them. You don't have that otherwise.

> Unless you have engineers constantly cross-cutting across the company, most engineers learn the style guidelines of a team/product/service by ramping up on the team.

Google uses a single repo and builds stuff from HEAD. This has up- and downsides, but one of the upsides is that its very easy to investigate unusual behavior in your dependencies. I was doing something similar just this week, and fixed obscure bugs in like 7 other teams tools that were causing downstream problems that those teams weren't aware of. No bureaucracy. I found the bug, fixed them, and sent the change to the owning team. Imagine other situations, where I'd need to maintain a fork, or file a bug to have them fix it, or coordinate an update with them.

You'll pay the cost no matter what, choosing to do so in a way that also provides value and mentorship makes a lot of sense to me.

> But promos drive _so much_ of the company culture, at least from what my partner tells me. And the video in the OP certainly makes it sound like Google has lots of bureaucracy, though I'm not sure how many of the requirements outlined in the video were there for hyperbole or not.

I mean the entire thing is tongue in cheek and again is a decade old. I joined Google ~5 years ago, and even then most of the problems with resource acquisition were solved ("flex") and borgmon and its associated readability were gone, replaced with monarch, which is centralized and used a python-based (though still admittedly arcane) query language that doesn't require readability or managing your own instance. And since then things have gotten even more turnkey for any service that's...reasonably shaped. And, well, the level to which promo drives company culture is consistently overstated (at least in some ways).

Consistency is what you get when you have a defined rule set programmatically enforced. If your looking for "readability" via human judgment, then you get a very different result.

A programmatically enforced set of rules is certainly one way to get consistency, but it isn't the only way. You can achieve consistency through culture and training too, and sometimes that's the only way.

Edit: you can look at Google's C-style guide for some examples, https://google.github.io/styleguide/cppguide.html#Structs_vs...

It isn't possible to statically analyze if a class/struct is a POD or if the methods enforce invariants. But it's often very easy to do so with a human eye. And there's value in the distinction!

Similarly, forcing someone to justify using a power-feature (operator overloading, templates, metaclasses, whatever) can only be done by a human. There may be cases where the power feature is warranted and the benefits outweigh the cost, but a linter can't know that. (and ultimately all of this comes back to: things look consistent, and when things are inconsistent, that's a strong signal that something unusual is happening and you should pay close attention)

What counts for readability is not set in stone by some language czar as the One True Way. Everyone knows the style guides can't be perfect which is why they're relatively mutable.

In any case, readability will comment on stuff that cannot easily be quantified, such as when to use a certain object hierarchy or dependency injection, etc...

That's not what readability is. There are plenty of automated tools that will give you results from running lint, ClangTidy and other tools. Readability is mostly about structuring your code well to be easily read. It's about architecting your code within a single file. It's about telling a junior SWE who reinvented the wheel use a library function he/she didn't know about instead.

So the rules can be codified sufficiently to test people on, but they can't be codified for a computer?

The only one that sounds more difficult to codify is telling people of the existence of duplicate functions. But as someone who contributes to the linux kernel, I can tell you right now that the only way that works reliably is to have a very large pool of reviewers. Very experienced engineers frequently miss what people are doing in other parts of the source base, the name might not be what they expect, etc, etc, etc. In the case of linux there are a fair number of duplicates, or similar functions, and people write coccinelle patches to replace them on a fairly regular basis after they have been in the kernel for years.

So, I doubt giving someone a formal gatekeeper flag, really helps vs just having wider change review.

I know of no automated tools available today that can determine if an identifier is accurately and usefully named. They can all tell if you are using the proper case, but that doesn't really tell you anything.

No tool like that tells you if returning a bool instead of an enum is appropriate here, or that a reference vs a pointer makes more sense given the rest of the code.

I'm sure a clever machine learning algorithm could figure that out with a corpus as large as Google's. Maybe. But no tool like that works today.

And not strangely at all, Google does accept "what clang-tidy does" as the canonical way of formatting text. But readability at Google is far more than just formatting.

Readability is frustrating and annoying, but more than just lint.

> I know of no automated tools available today that can determine if an identifier is accurately and usefully named.

Naming isn't very language-specific though [*], and poor naming should be flagged in coding interviews. Any Google engineer should generally have good naming, and should be able to critique others' naming in any language.

[*] Granted there are some language-specific naming conventions, like ! and ? in Ruby methods. But I'd say the majority of languages don't have important naming conventions, and those that do often have lints for them.

"poor naming should be flagged in coding interviews"

Exactly! And Readability process is exactly the training Google uses to ensure someone knows how to look for this issue, and that someone actually is looking for this issue in code reviews.

There is plenty about the Readability process I don't like (particularly that one has to get it through a fairly artificial process), but checking for things like the above (which I just use as one easily understood example) isn't something that happens automatically in an organization as large as Google, with as many engineers as Google.

A common story on HackerNews is just how bad the average programmer is, how crummy code they produce. Taking steps to ensure they do it better is a good thing.

We can argue about the process, for sure, and there are many things I would love to change about it. But "yeah, that should be done in code reviews" is exactly the idea here.

> Naming isn't very language-specific though [*], and poor naming should be flagged in coding interviews.

I'm not going to turn a candidate away because they use shorthand names when writing code under time pressure on a whiteboard.

This discussion wasn't about hiring but about code reviews. If you use bad names in your code then the reviewer is correct when they tell you to make better names.

> So the rules can be codified sufficiently to test people on, but they can't be codified for a computer?

I mean, given the halting problem alone, and the absence of general AI, yes, I'd say that...

I'm not sure what to tell you otherwise. I've worked at a place with such strong coding style guidelines, and even though I personally did not agree with every individual point, I accepted the tradeoff because being able to count on consistency in a very large code base was extremely helpful.

> So the rules can be codified sufficiently to test people on, but they can't be codified for a computer?

Small note: readability isn't a test or quiz you take (asterisk). It's obtained by merging code in the language you want readability for. If you merge code for a language often and the reviewers have very few style-based questions for the code then you will get readability fairly quickly.

> The only one that sounds more difficult to codify is telling people of the existence of duplicate functions. But as someone who contributes to the linux kernel, I can tell you right now that the only way that works reliably is to have a very large pool of reviewers. Very experienced engineers frequently miss what people are doing in other parts of the source base, the name might not be what they expect, etc, etc, etc.

A better example would be knowing when you should use `const std::string&`, `std::string_view` or `char*`. Example: https://abseil.io/tips/1

The best readability advice I have recieved has been:

1. Direct "I was confused by X" or "The recommended way to do A is using B", etc

2. Reasoned: "std::string_view is more efficient and clearer in intention than char ptr, it also improves type safety as it is read only and clear about ownership"

3. Linked to source material where examples are given totw or other examples in the code.

> Very experienced engineers frequently miss what people are doing in other parts of the source base, the name might not be what they expect, etc, etc

Very true. Readability can't help with that, nor is it designed to. It's mostly there to help novices and new hires. Experienced engineers already have readability themselves so they don't need this extra review.

Formatting and readability are two separate concepts (as other replies have pointed out). I'd like to specifically point to a fantastic example of what we mean when we say "readability": https://www.youtube.com/watch?v=wf-BqAjZb8M

Someone with readability in a language, who keeps up with the style recommendations, will generally produce code that is easier to read by other engineers.

The hypothetical discussion about readability is pointless.

Let's make it specific. Read https://google.github.io/styleguide/cppguide.html for readability for a language, namely C++. All the things that can be automated, automatic tools have been written for. But, for example, you can't automate "Prefer to use a struct instead of a pair or a tuple whenever the elements can have meaningful names." Because what does it mean for a name to be meaningful?

"Because what does it mean for a name to be meaningful? "

Are you optimizing for someone who already knows all the project lingo, or someone who doesn't know any of it?

Are your engineers native English speakers?

There are a whole bunch of things which make the perfect variable name frequently less than perfect, and putting project insiders in charge likely yields the opposite result.

Take: https://elixir.bootlin.com/linux/latest/source/mm/khugepaged...

If you don't know what a vma, pte, pfn, compound_page, young pte, huge page, lru, etc your going to be unable to even begin to understand what that code is doing, despite those all being pretty reasonable variable names and actually fairly industry standard concepts. It gets worse as you move to more esoteric topics. Expanding pte to PageTableEntry might help some subset of users, but at the expense of those that work on the code daily. So who do you optimize for? Is it readable if the only people that can read it already know what it does?

There's been a big investment in server platforms that strive to enable SWEs to build a new service that follows Best Practices with as little knowledge and handholding as possible. These consist of conformance tests that yell at you while you're coding if you are trying something generally thought to be bad, and semi-automated workflows that help you bring your code to production. When everything works as intended, the production workflows set up a decent set of alerts, acquire resources, configure CI/CD pipelines, and launch your jobs with just a few button presses on your part. (In practice, one of the steps will probably require debugging, but eh, it seems way better than the broccoli man video.)

Broadly speaking, there are tools to automate this or that, some technologies are getting deprecated and replaced by new ones

Also probably the privacy review could be a bigger bottleneck these days ;-)