News for hackers. Stuff that matters.
Re:A one-line change decreased our build times by 99% by mcv on Monday October 26, 2020 @06:07AM (#24894459)
On my first job, 20 years ago, we used a custom Visual C framework that generated one huge .h file that connected all sorts of stuff together. Amongst other things, that .h file contained a list of 10,000 const uints, which were included in every file, and compiled in every file. Compiling that project took hours. At some point I wrote a script that changed all those const uints to #define, which cut our build time to a much more manageable half hour.

Project lead called it the biggest productivity improvement in the project; now we could build over lunch instead of over the weekend.

If there's a step in your build pipeline that takes an unreasonable amount of time, it's worth checking why. In my current project, the slowest part of our build pipeline is the Cypress tests. (They're also the most unreliable part.)

Re:A one-line change decreased our build times by 99% by ravishi on Monday October 26, 2020 @06:24AM (#24894562)
At my second job in the industry I worked on a Python project that had to be deployed in a kind of sandboxed production environment where we had no internet access.

Deploys were painful, as any missing dependency had to be searched in our notebooks over 3G, then copied to an external storage, then plugged into a Windows machine, uploaded to the production server through SCP and then deployed manually over SSH. Sometimes we spent hours doing this again and again until all dependencies were finally resolved.

I worked there for almost a year, did many cool gigs and learned a lot. But my most valuable contribution came when at some point, tired of that unpredictable torture that were the deploys, started researching into solutions. I set up a pypi proxy into one of our spare office machines and routed all my daily package installs through that. Then I copied that entire proxy content into the production machine before every deploy, and voila, no more surprises.

I left this job a few weeks later, but have heard that this solution was very useful for many devs that joined the team afterwards.

Re:A one-line change decreased our build times by 99% by greesil on Monday October 26, 2020 @08:19AM (#24895305)
I suppose no Docker containers were allowed in prod either?
Re:A one-line change decreased our build times by 99% by ravishi on Thursday October 29, 2020 @07:32AM (#24929367)
Of course not. That was before docker, circa 2010. Our production environment was impossible to recreate.
Re:A one-line change decreased our build times by 99% by renke1 on Monday October 26, 2020 @06:53AM (#24894746)
> If there's a step in your build pipeline that takes an unreasonable amount of time, it's worth checking why. In my current project, the slowest part of our build pipeline is the Cypress tests. (They're also the most unreliable part.)

Would you say the (slow and unreliable) Cypress tests are worth it still?

Re:A one-line change decreased our build times by 99% by mcv on Monday October 26, 2020 @07:07AM (#24894839)
I don't know. We need some sort of e2e tests, and all e2e test frameworks are terrible in one way or another. Cypress is okay. I would prefer to only run it on production or the dev server and have alarms go off when they fail, but either the requirement is, or other developers have decided that it's necessary to pass all e2e tests before a feature branch can be merged into the master branch.

And I get the reason for it; you don't want to accidentally merge breaking changes. But it does make our build pipelines very slow and unreliable.

So are they worth it? I don't know. If I had my way, we'd only run them on master, and not make it a requirement for feature branches to pass them. Because if you fix one tiny thing, you now have to wait 15 minutes again for the Cypress tests to run. I think they'd be better in a different setup than what we're doing.

Re:A one-line change decreased our build times by 99% by pavon on Monday October 26, 2020 @11:40AM (#24897356)
We had similar issues with integration tests, and made them a separate jenkins job that didn't trigger automatically, but gitlab was still configured to require them to pass for merge. We would kick it off manually only after all other code review was complete. Then the only cases where we had to re-run it were the same cases where it would have failed in master if we only ran the test there, but it saved us the hassle of reverting or feeling pressured to get hotfixes into master quickly.
Re:A one-line change decreased our build times by 99% by smaps on Monday October 26, 2020 @09:41PM (#24903707)
Check out https://reflect.run/ as a replacement for Cypress. I started using it recently to do E2E testing at work in our staging environment to run a suite of tests before we move anything to production.

So far it's been great and has saved a couple of releases in a month or so of use!

Re:A one-line change decreased our build times by 99% by dzhiurgis on Monday October 26, 2020 @07:03AM (#24894817)
That's the nature of UI tests for most part. IIRC Cypress are written declarative tools which would make them even more unreliable and slow, albeit easier to fix.

Personally I've recently started using Playwright and I'm quite happy with it. There was occasional misunderstanding of their API, but 95% of time it's great. Microsoft is kicking butt these days.

Re:A one-line change decreased our build times by 99% by holtalanm on Monday October 26, 2020 @09:26AM (#24895884)
Cypress is horribly unreliable. We used to use it, and tests would pass, then fail on subsequent runs with no code changes, due to internal bugs within Cypress screenshot plugins, if I remember right.

I have no idea if it is any better now, but we dropped it about 6 months ago in favor of pure Selenium C# for our UI tests.

edit: a word

Re:A one-line change decreased our build times by 99% by holtalanm on Monday October 26, 2020 @09:20AM (#24895843)
> In my current project, the slowest part of our build pipeline is the Cypress tests

Oh man, I feel your pain.

Re:A one-line change decreased our build times by 99% by Cthulhu_ on Monday October 26, 2020 @10:05AM (#24896266)
Personally I think longer tests (like a full Cypress run) should not be a boundary to merging in prod if they take more than 10 minutes, but should be run nightly or continuously in the background.

I've not yet had the opportunity of having a large Cypress suite (working on it as we speak), but is it still more stable than e.g. Selenium is? Honestly 80% of issues we had with that were 'unstable' tests.

Re:A one-line change decreased our build times by 99% by mcv on Monday October 26, 2020 @10:12AM (#24896343)
Exactly. I would much prefer a setup like that over our current rule that all cypress tests must pass before merging.

A better rule might be that at least one unit or e2e test was added or updated to reflect the change in the code, and that that particular test succeeds. But run all the others on master.

One advantage (or occasional disadvantage) of Cypress test before merging, is that there is someone clearly responsible for fixing it if a test fails. Problem is, sometimes the failing test has nothing to do with anything the creator of the pull request did. It's still a mystery how that's possible, but it happens. Hence my feeling that Cypress tests aren't very reliable. At least some of ours aren't.

Re:A one-line change decreased our build times by 99% by holtalanm on Monday October 26, 2020 @10:11AM (#24896327)
Unfortunately, the issues we had with Cypress were with the framework itself, not the tests.

I used to write automation, and I can say that Selenium tests can be written to be very stable. Just depends on how they are written.