News for hackers. Stuff that matters.
Re:Ask HN: How do you work with Dependabot? by dynamite-ready on Friday August 12, 2022 @06:21AM (#32436507)
That would be a shallow reading, however. The last two major runtime issues wer actually one that broke the test runner and ignored a number of tests. And another runtime error was a Python Django specific sub dependency that broke the admin interface, which obviously, we don't explicitly test.

On the other hand, very recently, we had to abort a release, because of an outdated dependency that Dependabot DID actually raise.

Which is why I don't want to throw the baby out with the bathwater, as one or two people have suggested.

But I can say that I think that the reality of working with Dependabot, is not very well reflected in popular online articles.

Re:Ask HN: How do you work with Dependabot? by cr3ative on Friday August 12, 2022 @06:28AM (#32436548)
> the admin interface, which obviously, we don't explicitly test

This... is not obvious. It broke a part of your software which you care about. You care, it caused a problem, so it should be tested.

Re:Ask HN: How do you work with Dependabot? by dynamite-ready on Friday August 12, 2022 @07:08AM (#32436777)
You routinely write unit tests for dependent packages in your apps?
Re:Ask HN: How do you work with Dependabot? by manfre on Friday August 12, 2022 @07:41AM (#32437002)
This is not testing some other package. It's testing functionality provided by your app that happens to rely upon a 3rd party package. The 3rd party package has its tests, but doesn't know or care about the specific integration environment of your site.

With regards to writing tests of the Django admin, you need to do it if your site is customizing and/or depending upon it.

Re:Ask HN: How do you work with Dependabot? by numbsafari on Friday August 12, 2022 @08:38AM (#32437597)
No. This is what integration tests are for.

These kinds of failures are super annoying, but it’s the sort of thing that e2e and canary releases should help you get some coverage on.

Despite the marketing, unit tests are pretty fragile, especially in code with lots of deps (another good reason to limit deps)

Re:Ask HN: How do you work with Dependabot? by madeofpalk on Friday August 12, 2022 @09:38AM (#32438447)
Yes, we routinely write tests that interact with dependencies to make sure our application works.
Re:Ask HN: How do you work with Dependabot? by 4ndrewl on Friday August 12, 2022 @07:28AM (#32436901)
But once you've imported that package it's part of _your_ codebase.
Re:Ask HN: How do you work with Dependabot? by KronisLV on Friday August 12, 2022 @07:45AM (#32437030)
While this is true, many will probably disagree, just because they don't want to consider the maintenance burden that external dependencies will introduce.

So between choosing to write everything themselves (and getting nothing done), writing tests against dependencies (and getting little done due to the overhead), or claiming that external dependencies should have tests of their own, many will pick the latter.

Then again, in a world where create-react-app results in 180 MB of dependencies and about 1500 modules (probably different numbers now, using some older ones from my blog post), auditing security is an uphill battle, not even talking about actual testing.

The situation in the back end development, isn't that much better either, to be honest, because once you look into the complexity of any framework like Spring, Laravel, Django, Rails etc., it becomes apparent that creating a fully featured framework like that is a huge undertaking.

That said, you should at least test the bits where the external dependency is integrated with your codebase.

Re:Ask HN: How do you work with Dependabot? by dignati on Friday August 12, 2022 @06:46AM (#32436645)
> one that broke the test runner, and ignored a number of tests

That's unfortunate! For the project I'm working on, we've "solved" that by showing the number of test and the difference to the number of tests that ran on main.

FWIW, at previous jobs, upgrading Java dependencies was a major pain because they were all outdated and the latest versions introduced too many breaking changes for us. At my current job, we pretty much instantly merge all PRs from dependabot because we trust our CI. Upgrades rarely introduce problems and if they do, they are easy to fix.

Re:Ask HN: How do you work with Dependabot? by damienwebdev on Friday August 12, 2022 @06:27AM (#32436543)
> Python Django specific sub dependency, that broke the admin interface, which obviously, we don't explicitly test.

There's your problem.

Re:Ask HN: How do you work with Dependabot? by Narretz on Friday August 12, 2022 @06:35AM (#32436592)
Was it an update to the test runner or test specific packages that broke the test runner? I would ignore infrastructure/testability/tooling packages in dependabot and do them manually to prevent these errors.
Re:Ask HN: How do you work with Dependabot? by viraptor on Friday August 12, 2022 @07:48AM (#32437057)
Most test runners have an option (or can be easily modified) to fail when 0 or less than X tests have been run. You should use it for situations like this.
Re:Ask HN: How do you work with Dependabot? by lucasyvas on Friday August 12, 2022 @10:44AM (#32439432)
But what if there's also a bug where that feature doesn't work :)
Re:Ask HN: How do you work with Dependabot? by viraptor on Friday August 12, 2022 @12:22PM (#32440750)
What if there's a bug where your CI ignores all steps? There's always some situation possible and we can go all the way to nasal demons. You have to accept the risk at some point.