News for hackers. Stuff that matters.
Re:An ex-Googler’s guide to dev tools by tdeck on Thursday November 26, 2020 @02:52AM (#25217870)
> Many years ago, I did a brief stint at Google. A lot has changed since then, but even that brief exposure to Google's internal developer tools left a lasting impression on me.

Perhaps the operative phrase being "Many years ago". I currently work at Google, previously I worked at Square. Of the two, I generally prefer the OSS and off-the-shelf tooling at Square. Some things really are better at Google (code search and Blaze are definitely a big improvement over what we had), but many of our monitoring and CI tools feel antiquated and cluttered with inscrutable debris. Or take Gerrit, where each change has 3 different IDs and the developers decided that every workflow status should be expressed as an integer in the range [-2, 2] for some reason [1].

They were probably amazing (and much simpler) tools back in the day, but the world has moved and we're somewhat constrained by what's familiar.

[1]: https://gerrit-review.googlesource.com/Documentation/images/...

Re:An ex-Googler’s guide to dev tools by matt_o on Thursday November 26, 2020 @02:09PM (#25222759)
> I currently work at Google, previously I worked at Square. Of the two, I generally prefer the OSS and off-the-shelf tooling at Square.

I wrapped a 2-year stint at Facebook recently and my experience was similar to yours.

Some tools were amazing and cool and I really appreciated how they evolved over time to manage huge amounts of resources. But most were subpar when compared to OSS tooling - outdated, deprecated functionality, very little documentation, very few people working on them. The common approach was to learn about "the duct tape" way to make something work, then pass it on to new engineers.

An example would be tool X for working with diffs (PRs). It's the latest and greatest, except that it only covers 75% of its predecessor's, tool Y's, functionality, so you end up learning both. Tool Y has been "deprecated" for the past 3-4 years. Some of its features don't work, but you'll only know when you try and execute them.

Re:An ex-Googler’s guide to dev tools by hinkley on Thursday November 26, 2020 @03:00PM (#25223101)
You have to make some allowances for first movers. The designs of new tools are often ill conceived because they have no templates to steal ideas from. But the emotional investment often keeps them from switching to better tools because We’ve Always Done Things This Way.

The thing that bugs me most about my company’s tools are the ones that are enough younger than OSS solutions that someone either didn’t look very hard, or didn’t want to find anything (so they could write their own). Other priorities come along and those tools eventually can’t keep up with OSS alternatives, but the apologists take over.

You will not continue to get accolades for tools you wrote three years ago. The only “value” you derive is the time and effort it saves, offset my the effort expended. The time and effort expense of external tools are often lower. And, when the tools are annoying, you can commiserate with your coworkers instead of being the target of their criticisms. Which is often under-appreciated.

Re:An ex-Googler’s guide to dev tools by kortilla on Thursday November 26, 2020 @10:30AM (#25220806)
> in the range [-2, 2] for some reason [1].

This was actually pretty useful in OpenStack back in the day. The reviewers who couldn’t yet approve code could only put +1s on there (and there were a lot of low quality reviewers who slapped these everywhere) so it was very obvious when a patch still needed attention from a commit-privileged dev.

Then -1 was standard review feedback of stuff that needed improving and -2 would come from commit-privileged devs when the patch fundamentally didn’t fit with the project direction.

Re:An ex-Googler’s guide to dev tools by klenwell on Thursday November 26, 2020 @01:51PM (#25222629)
Yes, that range looked odd to me at first glance. But your explanation made me realize that I actually use it all over the place. Except on a 1-5 scale. Starting with good enough at 3 (or 0 in this case):

1 = WTF?

2 = Needs Improvement

3 = Good Enough

4 = Better than Good Enough

5 = Almost Perfect

Daniel Kahneman recommends a scale like this for evaluating job candidates across core competencies. It's a surprisingly powerful little heuristic.

Re:An ex-Googler’s guide to dev tools by generalizations on Thursday November 26, 2020 @03:38PM (#25223346)
The [-2,2] scale made immediate sense to me because it's zero-indexed:

-2 = WTF

-1 = Needs Improvement

0 = Good enough

1 = Better than Good Enough

2 = Almost Perfect

It's even intuitive, since anything above zero is 'above and beyond' and anything below zero isn't good enough yet.

Re:An ex-Googler’s guide to dev tools by jdbernard on Thursday November 26, 2020 @01:31PM (#25222427)
The system makes sense but it feels like there is probably a more intuitive easy to express those than with numbers. Naming is hard, of course, but off the top of my head maybe something like

-1: Changes Requested

-2: Declined

1: Approved

2: Accepted

Re:An ex-Googler’s guide to dev tools by antonvs on Friday November 27, 2020 @12:31AM (#25225763)
Why would that matter? This is for developers, not English majors.
Re:An ex-Googler’s guide to dev tools by jdbernard on Friday November 27, 2020 @05:22PM (#25233087)
I didn't down vote you, and your question deserves a serious reply because the same principle is broadly useful.

Communication is hard. People actively in the team probably have the context to understand the differences, but that implied context is what tribal knowledge is made of.

OP indicated that there were non-trivial "non-linear" differences going from 1 to 2 and -1 to -2. The use of numbers implies a relationship that doesn't really exist. The use of accurate language makes the actual meaning immediately obvious without the need for implied context. For example, I had no idea that only privileged devs could give 2s. "Accepted" and "Declined" do imply a finality that more accurately mirrors the intended usage.

This may seem overly analytical for something that is easy to just explain, but this kind of thing build up in layers.

Re:An ex-Googler’s guide to dev tools by tdeck on Saturday November 28, 2020 @08:50PM (#25242276)
Exactly this. It's not as if two +1s equals a +2 (which one night intuitively think based on the nomenclature).
Re:An ex-Googler’s guide to dev tools by ma2rten on Thursday November 26, 2020 @03:05AM (#25217933)
Note that Critique (Google's main code review UI) doesn't have the same issues that Gerrit apparently has.
Re:An ex-Googler’s guide to dev tools by cmrdporcupine on Thursday November 26, 2020 @07:37AM (#25219460)
I work at Google and mainly use Gerrit because of the products I work on and yes, Critique is miles better.
Re:An ex-Googler’s guide to dev tools by corentin88 on Thursday November 26, 2020 @04:17AM (#25218298)
Critique seems a well-guarded secret inside Google. Can't find more information on it. No images. Interesting
Re:An ex-Googler’s guide to dev tools by lima on Thursday November 26, 2020 @07:59AM (#25219640)
Not really. The "Software Engineering at Google" book[1] has a whole chapter on it with screenshots.

Gerrit, Reviewable and Phabricator are very similar, except that Critique is much more polished.

Having used all three, Gerrit works best and the new UI is starting to look pretty good. Reviewable has great UX, but inherits some of the disadvantages of GitHub PRs.

[1]: https://books.google.de/books/about/Software_Engineering_at_...

Re:An ex-Googler’s guide to dev tools by hadrien01 on Thursday November 26, 2020 @01:06PM (#25222234)
Your link just goes to the Google Books homepage in my browser
Re:An ex-Googler’s guide to dev tools by kubanczyk on Thursday November 26, 2020 @03:31PM (#25223315)
Same for me. I think it's https://books.googlecom/books?id=WXTTDwAAQBAJ. It has no preview of the Critique-related fragments.
Re:An ex-Googler’s guide to dev tools by Sesse__ on Thursday November 26, 2020 @01:00PM (#25222191)
It's a reimplementation of an earlier system called Mondrian, which you can find some information on.
Re:An ex-Googler’s guide to dev tools by cmrdporcupine on Thursday November 26, 2020 @02:26PM (#25222869)
I used Mondrian for the last couple months it existed, and I recall critique being fairly different.

I think they kept the same keystrokes, though.

That is one nice thing about Critique is although it's a webapp you can drive it pretty much entirely by keyboard. It has the feel of an old usenet newsreader like nn, in a way.

Re:An ex-Googler’s guide to dev tools by ikiris on Thursday November 26, 2020 @04:53AM (#25218483)
Here I'm sitting here thinking could I draw it from memory?
Re:An ex-Googler’s guide to dev tools by antoinealb on Thursday November 26, 2020 @05:03AM (#25218529)
It went through a re-design recently. I don't think I would call it a well-guarded secret though, if you look at Phabricator's Differential, it is very similar [0].

[0] https://www.phacility.com/phabricator/differential/

Re:An ex-Googler’s guide to dev tools by jonhohle on Thursday November 26, 2020 @10:09AM (#25220619)
Reading through that page it seems like every single component is an unrelated, seemingly random, “cute” name. Are cli, triggers/actions, and rules too generic? Out of context, what meaning do Arcanist, Maniphest, or Herald have?
Re:An ex-Googler’s guide to dev tools by chrizel on Thursday November 26, 2020 @10:35AM (#25220844)
Related answer from one of the developers: https://www.quora.com/Phabricator/Phabricator-Why-is-the-Bro...

But yeah... even I find the names difficult to handle and remember, even after using Phabricator for more than 4 years now. Nevertheless, it is a great piece of software, especially the code review process is top-notch.

Re:An ex-Googler’s guide to dev tools by rockwotj on Thursday November 26, 2020 @06:03AM (#25218857)
I agree - grafana is wayyy better than viceroy and the other monitoring dashboards that were available when I worked at Google.
Re:An ex-Googler’s guide to dev tools by jeffbee on Thursday November 26, 2020 @10:44AM (#25220930)
Interesting. Viceroy can put way more pixels on the screen than grafana with the same cpu and memory usage, and panopticon is specifically written to send little traffic over the wire, so it’s useful to an oncall engineer over a poor mobile backhaul like GPRS. Grafana is a massive resource hog in all dimensions.
Re:An ex-Googler’s guide to dev tools by rockwotj on Thursday November 26, 2020 @12:24PM (#25221890)
Hasn't pcon been deprecated for a year and a half now? :P

More pixels != to more information. The visuals and varieties in ways to present data in grafana always seemed more intuitive for me, being able to lay stuff out in different ways was great. I worked on a G-on-G project at Google and used both grafana and viceroy. I never had an issue with grafana on mobile, but I was never was able to see viceroy on a mobile browser because of the Corp network.

Re:An ex-Googler’s guide to dev tools by jeffbee on Thursday November 26, 2020 @02:35PM (#25222926)
On viceroy if you unintentionally put 2000 lines on a graph, you get a visual mess. If you do that on grafana your tab will hang for ten minutes. Big difference.
Re:An ex-Googler’s guide to dev tools by rockwotj on Thursday November 26, 2020 @03:15PM (#25223205)
True, but that only happens when setting up graphs. Graphs are not changed as often as they are viewed.

I'm sure viceroy is more performant, and I'm not denying that, I may also be scarred a bit because I found writing mash queries an arcane nightmare, and those are related to viceroy.

The default (rpc latencies and etc) views and metrics you get are amazing however.

Re:An ex-Googler’s guide to dev tools by joshuamorton on Thursday November 26, 2020 @03:26PM (#25223276)
You can also get tons of lines when modifying group by expressions, which you can do ineffectively.

Mash is indeed arcane, although gmon (viceroy) makes it worse than usual.

Re:An ex-Googler’s guide to dev tools by lima on Thursday November 26, 2020 @03:30AM (#25218058)
Gerrit is configurable and it is common to use it like Critique with blocking comments and LGTMs.

It's still a big step up from, say, GitHub PRs, and the UX has improved significantly.

Re:An ex-Googler’s guide to dev tools by daralthus on Thursday November 26, 2020 @06:56AM (#25219163)
> They were probably amazing (and much simpler) tools back in the day, but the world has moved and we're somewhat constrained by what's familiar.

Exactly my feelings about Phabricator. GitLab simply runs circles around it if you consider the issue tracking, CI etc. as part of the problem.

Re:An ex-Googler’s guide to dev tools by wheaties on Thursday November 26, 2020 @01:27PM (#25222396)
We moved from Phabricator to JIRA, Github and CircleCI. We couldn't be happier. It's one thing to have a single tool with 70% of the functionality you need, it's another to have 90%+. That 20% makes such a huge gulf it's not even funny. We were able to ditch several thousands of lines of cobbled together glue that moved us 10% forward but added overhead for any new project which set us back a week or more.
Re:An ex-Googler’s guide to dev tools by Cthulhu_ on Thursday November 26, 2020 @07:10AM (#25219247)
I would've loved to have Gitlab at our company, but unfortunately we're on a tight budget so they didn't want to go for anything paid.
Re:An ex-Googler’s guide to dev tools by JeremyNT on Thursday November 26, 2020 @08:01AM (#25219656)
Gitlab CE would be perfect, then, since it's free. If you're on a tight budget it actually seems like one of the best options available.
Re:An ex-Googler’s guide to dev tools by paulryanrogers on Thursday November 26, 2020 @09:54AM (#25220495)
Tried self hosting once and it was challenging just to get setup. That said their free hosted solution is feature such enough for most of my projects.
Re:An ex-Googler’s guide to dev tools by djeiasbsbo on Thursday November 26, 2020 @11:02AM (#25221105)
I self host it at home. Haven't had problems but probably only because of how small my instance is (10 people). The omnibus installer makes it relatively simple to maintain!
Re:An ex-Googler’s guide to dev tools by dboreham on Thursday November 26, 2020 @09:01AM (#25220067)
> integer in the range [-2, 2] for some reason

This sounded awesome to me (imagining some Fortran code behind it), but disappointingly the cited screen shot only has the range [-1,0,1] which is hardly the same.

Re:An ex-Googler’s guide to dev tools by sershe on Saturday November 28, 2020 @09:56PM (#25242593)
I think this might come from a typical feedback on patches in Apache (or all of OSS) - -1 rejected as is, +1 approved. I've also seen -0 (don't approve but won't block/reject) and +0 (ok with it but didn't review/won't approve)
Re:An ex-Googler’s guide to dev tools by broeng on Thursday November 26, 2020 @11:08AM (#25221162)
-2 and 2 can be disabled, which I guess is the case on the screenshot you saw.
Re:An ex-Googler’s guide to dev tools by raverbashing on Thursday November 26, 2020 @05:27AM (#25218674)
Yes please, Github PRs are much, much better than Gerrit or Ritveld (urgh), and the +1/+2 quirks of it

No, I'll take Github PRs any day. They could be better, of course.

Re:An ex-Googler’s guide to dev tools by u801e on Thursday November 26, 2020 @09:56AM (#25220509)
Github PRs (at least on the enterprise version) still have the problem of collapsing comments where the associated line of code changed in some way that had nothing to do with the comment. They also require a lot of scrolling if the overall diff is rather long. And, they still don't provide a way to comment on commit messages themselves.
Re:An ex-Googler’s guide to dev tools by plorkyeran on Thursday November 26, 2020 @12:32PM (#25221967)
Collapsing comments when the code changes is the correct thing to do 90% of the time in my experience, and for that other 10% you can manually mark is as unresolved. It's the correct default IMO.
Re:An ex-Googler’s guide to dev tools by u801e on Thursday November 26, 2020 @02:33PM (#25222911)
> Collapsing comments when the code changes is the correct thing to do 90% of the time in my experience

Why not just leave them uncollapsed and allow one to mark them as resolved when the change is actually resolved. The other problem is that it's difficult to see what actually changed when a line is marked out of date without having to scan through the entire updated diff and then jump back and forth between the diff and conversation view.

Collapsing them by default makes it difficult to find the comment by scanning. If they were uncollapsed until I marked them as resolved, then I could at least use the find feature in the browser and search for my username to find my comments.

Re:An ex-Googler’s guide to dev tools by raverbashing on Thursday November 26, 2020 @10:13AM (#25220656)
Agreed on the comment collapse, about comment on the commit message I'd say it's a commit on the main PR (since by the time the PR is done the commit msgs are pretty much not changeable, except for the last one)
Re:An ex-Googler’s guide to dev tools by u801e on Thursday November 26, 2020 @02:35PM (#25222927)
> about comment on the commit message I'd say it's a commit on the main PR

Except that if the overall diff line I choose to comment on about the commit message changes in some way, then the comment is collapsed, which makes it difficult to keep track of what needs to change.

> since by the time the PR is done the commit msgs are pretty much not changeable, except for the last one

They can be changed via a git rebase.

Re:An ex-Googler’s guide to dev tools by madhadron on Thursday November 26, 2020 @11:51AM (#25221555)
Stacked reviews are the crucial thing missing from GitHub pull requests. Truly, once you get used to working wit hthose, going back is intolerable, to the point where I now have a hacked together workflow where I live in git rebase -i and have it automatically push specially named branches for each commit in a stack.
Re:An ex-Googler’s guide to dev tools by ericyu3 on Saturday November 28, 2020 @07:50PM (#25241948)
So a few months ago I actually found a tool for this! The tool makes it easy to manage stacked reviews on GitHub. It's working super well for me. I also showed it to a bunch of people I work with, and a lot of them have taken it up.

Link: https://github.com/ezyang/ghstack

If you're interested, I'm happy to talk you through it - just book some time here: https://calendly.com/ericyu3/15min

Re:An ex-Googler’s guide to dev tools by u801e on Thursday November 26, 2020 @02:59PM (#25223093)
At my place of work, we've sort of implemented that feature by using fixup or squash commits. That is, if a comment is made on a diff line in a PR, use git blame to determine which commit was responsible for that line of code, update the line and commit it with a commit title like: fixup! <original commit title>. If we want to update the commit message, we use "squash! <original commit title>" and put the updated commit message in the commit message body (using the --allow-empty switch when committing so that we don't have to change any code to make a commit).

Then, we can run git rebase -i --autosquash --keep-empty to handle applying the changes requested to the correct commits at the end of the review process.

Re:An ex-Googler’s guide to dev tools by selestify on Thursday November 26, 2020 @01:07PM (#25222240)
What are stacked reviews?
Re:An ex-Googler’s guide to dev tools by erincandescent on Thursday November 26, 2020 @05:10PM (#25223885)
You have a task to build a Thing, which involves creating the basic "Thing framework" and then implementing subfeatures A, B and C of into/on top of it

Maybe you'll build the framework & subfeature A together, because you need some bit of meat in there to properly figure things out and be able to test things

On Gerrit you'd probably create two commits which get pushed as two CLs

* CL1 Create Thing framework * CL2 Implement A for Thing

They're now pushed for people to review. While they're looking at them, you continue working on, making CL3 implementing B

* CL3 Implement B for Thing

One of your coworkers points out an issue in CL1, so you fix it (by amending the commit) and repush the stack. Now your stack is

* CL1 Create Thing framework [v2] * CL2 Implement A for Thing * CL3 Implement B for Thing

CL1 & CL2 get approved so you merge them (though typically with Gerrit they're cherry picked - CLs are individual Git commits). You push up an implementation of CL4, so your stack now looks like

* CL3 Implement B for Thing * CL3 Implement C for Thing

The important point is that CLs are atomic (and are reviewed atomically) even if they depend upon each other (i.e. are a part of a stack). When you're working in Git you typically (unless for whatever reason you have multiple unrelated stacks on the go, which is relatively rare) just work off of the master branch, so all the commits between `origin/master` and `master` - i.e. the set that automatically show up in `git rebase -i` and similar tooling - are your stack. When you pull, you `git pull --rebase` (or rather set the config for your machine or repo to default to that). When you've revised something in your stack or want to add something to it, you just do "git push HEAD:origin/refs/for/master" update Gerrit with the latest version of your stack

It takes a little time to get used to (and you have to learn how to hold `git rebase -i` properly), but when you're used to it it's immensely more productive than the Github/clones PR flow (and doesn't involve manual branch juggling, etc). I can't express just how badly they deal at handling reviews of stacked PRs - either you create your stacked PR targeting master (and it gets all of the changes from the underlying PRs merged into its changes list, which makes reviewing it harder) or you target it at the branch of the underlying PR (which is non-obvious and painful and you have to manually remember to shift it across when the PR beneath it gets merged)

When I'm in this situation when dealing with something which is PR based, I tend to end up merging CL1 & CL2 (because its easier to review things when you have an example) and hang on to CL3 & CL4 on my local machine until the first one gets merged

Re:An ex-Googler’s guide to dev tools by u801e on Thursday November 26, 2020 @11:37PM (#25225579)
> The important point is that CLs are atomic (and are reviewed atomically) even if they depend upon each other (i.e. are a part of a stack).

What prevents a situation like CL2 getting approved and merged prior to the resolution of CL1? Or more generally, what ensures that the ordering of commits in a set of CLs in a given stack is preserved prior to merging?

> I can't express just how badly they deal at handling reviews of stacked PRs - either you create your stacked PR targeting master (and it gets all of the changes from the underlying PRs merged into its changes list, which makes reviewing it harder) or you target it at the branch of the underlying PR (which is non-obvious and painful and you have to manually remember to shift it across when the PR beneath it gets merged)

Rather than stacked PRs (which essentially doubles the number of commits since Github (and possibly Gitlab) introduces a merge commit for each PR even if it's a fast-forward merge), would it not be better to just combine CL1 through CL4 into a single feature branch where each commit corresponds to a CL? It would make for a large PR, but it can be reviewed on a per commit basis.

Re:An ex-Googler’s guide to dev tools by Quekid5 on Thursday November 26, 2020 @01:49PM (#25222610)
Yes, stacked reviews absolutely are a game-changer for a sane commit history and iterating patches over a few rounds of review.