News for hackers. Stuff that matters.
Re:Changing how I review code by yjftsjthsd-h on Wednesday June 30, 2021 @02:04PM (#27691657)
> Another common critique is that the pull requests encourage reviewers to only look at the code. They never even pull down the changes they are reviewing!

You should run it somehow, but if you've got CI running tests, which I think you should have anyways, then does it matter?

Re:Changing how I review code by kmtrowbr on Wednesday June 30, 2021 @05:48PM (#27693770)
You should attack it. You should pick it apart and try to break it in as many ways as possible. Of course you should meter your effort depending on how critical the changes are. It amazes me the lackadaisical attitude towards code review. "It will probably be okay" is the worst attitude to have with computer code. It is the most brittle, brutal environment imaginable.
Re:Changing how I review code by LeftHandPath on Wednesday June 30, 2021 @08:55PM (#27695257)
It's something I've noticed in modern peer-review practices, too. And some have gone as far as creating fake papers to study whether or not they get published: https://www.theatlantic.com/ideas/archive/2018/10/new-sokal-...

There's a human instinct towards laziness. If there are no outwardly-visible reasons to distrust something, it doesn't get evaluated. It's why a hard-hat and a clipboard can get you backstage, and why you don't spend 2 minutes scanning every bush for crouching predators before you walk past it. In everyday life, a "lazy" tendency saves time by avoiding needless work. In engineering, academia, etc., it can create dangerously low standards.

Re:Changing how I review code by all2 on Wednesday June 30, 2021 @05:50PM (#27693796)
Can you expand on this?
Re:Changing how I review code by kmtrowbr on Wednesday June 30, 2021 @05:59PM (#27693887)
You should pull the PR you're reviewing down into your IDE and you should spend like, an hour reviewing it carefully and trying to break it. Have a mental checklist: say they renamed a function ... is the file named the same as the function? Did they remember to rename the file? Did they delete parts of the code? Did they remember to delete everything related to that? Often the most challenging parts are negative: it's harder to remove properly than it is to add new things. Say they changed the validation on a model: is this going to work well with the production database? Do we need to write a migration to catch the database up to the new reality of the code? Automated tests are just automated warning flags that give you confidence that the codebase is basically hanging together. You still need to be very careful & thoughtful as you're making changes.

I like to actually reset the branch so I can see all the raw changes in my IDE, this helps me to think it through, in git you can do this via:

git pull feature-branch; git checkout develop; git checkout -b my_initials/feature-branch; git merge feature-branch --squash --no-commit

Re:Changing how I review code by QuercusMax on Wednesday June 30, 2021 @08:15PM (#27695025)
How big are these changes? That sounds horrendously slow.

If you don't trust your tests, write better tests.

If you don't trust your migrations, write better tests.

Re:Changing how I review code by Dunedan on Thursday July 01, 2021 @01:43AM (#27696588)
In probably most cases the person who does the code changes does write tests for it as well. So as a reviewer without understanding what the code and the tests do, how do you want to assess if the tests actually do something useful?
Re:Changing how I review code by QuercusMax on Sunday July 04, 2021 @02:38PM (#27731679)
If your code and tests are so opaque that the reviewer can't understand them without physically running and modifying the code, then as a reviewer you should ask the author to make it clearer.

Because otherwise that's how you get write-only code. Things like code style and documentation can make a big difference, and review time is when you can help enforce them and build team culture and knowledge.

Re:Changing how I review code by efficax on Wednesday June 30, 2021 @07:53PM (#27694887)
An hour is way too much time. If you need that long the pr is too big and needs to be broken up into smaller parts
Re:Changing how I review code by juanca on Wednesday June 30, 2021 @02:16PM (#27691792)
At least with UI code, the tests / code don't really express the experience for the user. (But it's getting better with time!)

In that practice, it's better to run it locally! For each UI file / function, try to get it to execute from a user's POV.

Re:Changing how I review code by loloquwowndueo on Wednesday June 30, 2021 @04:49PM (#27693292)
A reviewer in my team is fond of changing some of the code in the PR and running the tests. The expectation is that if the code is changed randomly a test should always fail. This helps catch untested code paths.

It’s somewhat similar to the practice of automated code fuzzing, only manual and scoped to changes in a particular PR.

Re:Changing how I review code by travisjungroth on Wednesday June 30, 2021 @05:17PM (#27693500)
That's mutation testing[1]. For most languages there are tools that can scope the mutations to a diff.

[1]https://en.wikipedia.org/wiki/Mutation_testing

Re:Changing how I review code by all2 on Wednesday June 30, 2021 @05:49PM (#27693785)
Just because there are tests doesn't mean the tests are any good. I've only started my quest in TDD and I've discovered it is rather trivial to write tests that don't really mean anything.

I find myself asking "what is this supposed to do?" and "does this actually do what it is supposed to do?". I tend to drift into a design mindset during test writing (more-so than in code review) that highlights data flow and the basic bits of logic. Even then, I find that I can miss test cases rather easily.

Re:Changing how I review code by jimbobimbo on Wednesday June 30, 2021 @06:44PM (#27694379)
That's what code coverage tooling + human code reviewers are for. Look at the change in product, look at the amount of changes in tests - do they make sense? E.g., product change adds a non-trivial function with 10 if/else branches, but UTs touch only happy path - that's a legitimate blocker on the PR.
Re:Changing how I review code by sfvisser on Wednesday June 30, 2021 @04:24PM (#27693045)
Of course it matters. Otherwise, how do you know if your app works as intended?

And don’t tell me your tests tell you, because I guarantee you they don’t!

Re:Changing how I review code by the_af on Wednesday June 30, 2021 @05:34PM (#27693644)
Isn't that what QA/testers are for? Testing that the feature/release works as intended?
Re:Changing how I review code by elwell on Wednesday June 30, 2021 @05:35PM (#27693659)
In practice, I end up finding at least one bug nearly every time I pull the code down and test it locally.
Re:Changing how I review code by watwut on Wednesday June 30, 2021 @03:01PM (#27692278)
That is what testers are for. Testing is great thing.
Re:Changing how I review code by bern4444 on Wednesday June 30, 2021 @03:32PM (#27692570)
/s/testers/users

Just kidding and in reality UI tests exists and aren't super difficult to set up. The front end has plenty of tools like Cypress and its pretty simple to automate running a bunch of tests that screenshot and diff compare your site.

I never really pulled down UI code to test them out. There has to be a certain level of trust between engineers. Sometimes I'd ask or post screenshots of changes but that was rare.

Re:Changing how I review code by akiselev on Wednesday June 30, 2021 @06:12PM (#27694045)
> Just kidding and in reality UI tests exists and aren't super difficult to set up. The front end has plenty of tools like Cypress and its pretty simple to automate running a bunch of tests that screenshot and diff compare your site.

In a well oiled machine, that's what the QA department does. They have close working relationships with the engineering and customer support departments so they develop testing processes and systems tailored to the "quirks" of engineers and users. They spend extra time on the places they know the engineers have blind spots and work on automated regression tests based on customer support workloads.

Whether they go as low level as unit tests depends on the organization but one place I worked at, the domain was complicated enough that "I see no red squiggles in my IDE, pinky swear" was the threshold to go from engineering to QA. It was an ongoing joke that whenever there was a major dependency update, 10-25% of the work sent to QA that day contained obvious syntax errors and wouldn't compile because the IDE everyone used often took an hour or more to reindex the project without any clear indication that it was in a useless intermediate state. Running the compiler would slow it and the indexer to a crawl, so the engineers just YOLOd it so they could move on to other work, and the QA department set up a static analyzer with an empty rule set on all PRs to catch dumb errors caused by immutable process/purchasing decisions.

IME these kinds of QA departments are the #1 springboard into software engineering roles because they work so closely with engineering and are exposed to code in the form of automated tests.

Re:Changing how I review code by shinryuu on Wednesday June 30, 2021 @04:22PM (#27693018)
Still, getting a second set of fresh eyes using the code that you wrote is pretty invaluable. This can then be fed back as improvements.