News for hackers. Stuff that matters.
Re:How to fuck up software releases by vortico on Sunday October 13, 2019 @10:55PM (#21244906)
Here's a somewhat redacted version. `m` is my alias for `make`.

    - Finalize source
     - Legal check `LICENSE-dist.txt`
     - Update `CHANGELOG.md`
     - Bump version in `Makefile` and `Core.json`
     - Commit "Bump version".
    - Build (three OS's can be done in parallel)
     - `m clean`
     - `git pull`
     - Make sure you have the latest `Fundamental.zip` package in source root.
     - `m dist`
     - Manually test installer and fragile features (audio drivers, patch loading) for ~10 minutes.
     - `m notarize` (on Mac)
     - `m upload`
     - `git tag vX.Y.Z`
     - `git push --tags`
    - Release
     - Update version title and URLs in `Rack.pug`
      - `m upload`
      - At this point, normal users have access to new version.
     - Update server version in `config.coffee`
      - `m restart`
      - At this point, normal users will swarm to download new version. Keep an eye on server bandwidth.
    - Publicize
     - Twitter https://twitter.com/home
     - Facebook https://www.facebook.com/vcvrack/
      - Share on group
     - Forum https://community.vcvrack.com/c/announcements
Re:How to fuck up software releases by jl-gitlab on Monday October 14, 2019 @12:51AM (#21245343)
Interesting list, thanks for sharing. I'm the PM for CI/CD at GitLab, and I don't mean to hijack the topic, but wanted to give a heads up that we are looking at building a feature for these kinds of procedures that aren't quite pipelines, based on Jupyter Runbooks. Would love to hear your feedback: https://gitlab.com/gitlab-org/gitlab/issues/9427
Re:How to fuck up software releases by vortico on Monday October 14, 2019 @01:34AM (#21245498)
Currently the most difficult and most time consuming step in the release checklist is finding space on my desk for three laptops so I can use them all simultaneously. Releasing only takes 1-3 hours, and I unfortunately wouldn't want to spend hours learning and setting up automated methods to perhaps shave 10 minutes off the process.

Manual releasing allows me to iteratively improve the process and spot possible issues, whereas any automation would reduce build reliability.

Re:How to fuck up software releases by judge2020 on Monday October 14, 2019 @02:14AM (#21245643)
> whereas any automation would reduce build reliability.

I guess this depends on what your release process is; for example, most npm libraries benefit from CI that runs the build script and runs `npm version $TAG && npm publish`. There's not much that could be improved here, especially if the build script is just an alias for running `webpack`.

Re:How to fuck up software releases by scarejunba on Monday October 14, 2019 @12:55AM (#21245357)
There's a method I read about on lobste.rs for this. You write down the list as you did and then turn pieces into "Do X by running Y" and over time more things get automated.

That's a hefty deploy protocol, though, and automating cross-platform builds may be more trouble than it's worth if you're releasing rarely.

It's pretty neat you follow a deploy checklist.

Re:How to fuck up software releases by rgoulter on Monday October 14, 2019 @06:08AM (#21246595)
Related to the process of "write down what you did", I'm reminded of this article about "literate sysadmin/devops" which used Emacs' org-mode as a way of combining prose, commands and command output. http://howardism.org/Technical/Emacs/literate-devops.html
Re:How to fuck up software releases by maccard on Monday October 14, 2019 @06:34AM (#21246717)
I read you should write it as an interactive script e.g.:

    input("Build X")
    input("Build Y")
    input("Zip X & Y")
    input("Upload release")
THen run the script and press enter as you do the steps. It means if you're interrupted as your proceed, you can resume exacly where you were before.

If you implement it as functions:

    def buildX():
        input("Build X")
    def buildY():
        input("Build Y")
    def ZipUp():
        input("Zip X & Y")
    def Upload():
        input("Upload release")
    if __name__ == '__main__':
        buildX()
        buildY()
        ZipUp()
        Upload()

Then you can automate parts piecemeal.
Re:How to fuck up software releases by enriquto on Sunday October 13, 2019 @11:17PM (#21245004)
this list is a very simple algorithm. Why don't you tun it into a script deploy.sh ?
Re:How to fuck up software releases by tbrownaw on Monday October 14, 2019 @12:01AM (#21245182)
Some of the steps look like they'd be hard to do that for. Like for example the one about making sure the license is correct, and the one about testing parts that apparently don't have good automated tests / interact with the real world.
Re:How to fuck up software releases by enriquto on Monday October 14, 2019 @12:27AM (#21245265)
> Some of the steps look like they'd be hard to do that for.

Then those steps must be removed. I thought the mantra "deploy is ONE step" was a more or less universally acknowledged truth.

Re:How to fuck up software releases by creatornator on Monday October 14, 2019 @12:40AM (#21245305)
I mean unless a computer can check that all dependencies have compatible licenses, that is unrealistic. It's a necessary step for publicly released software to avoid legal issues
Re:How to fuck up software releases by cyphar on Monday October 14, 2019 @12:45AM (#21245316)
Linux distributions (though I can only speak to openSUSE's process) have automated scripts (which I believe are written by our lawyers) which check whether the license of a package matches the license of the files inside the package. It's how most package legal review gets done (and if the script can't figure it out, it gets escalated to our actual lawyers too review. You cannot submit a package to any one of the distributions we ship without the legal review being approved.

So it is clearly possible to do -- and there are all sorts of tools which figure out what SPDX license entries apply for every dependency (or vendored dependency) of a given project.

Re:How to fuck up software releases by udkyo on Monday October 14, 2019 @03:48AM (#21246025)
I took the parent posts to mean that step involved checking the project license, not its dependencies.

For the latter though, Fossology, Scancode etc. can help.

[dead]
Re:How to fuck up software releases by yjftsjthsd-h on Monday October 14, 2019 @01:26AM (#21245471)
Unfortunately, not everyone acknowledges this truth;)
Re:How to fuck up software releases by antoinevg on Monday October 14, 2019 @05:30AM (#21246424)
…or has a sense of humour about it! ;-)
Re:How to fuck up software releases by gdfasfklshg4 on Monday October 14, 2019 @05:36AM (#21246455)
Because it involves inherently manual processes.
Re:How to fuck up software releases by vortico on Sunday October 13, 2019 @11:35PM (#21245076)
No it's not.
Re:How to fuck up software releases by enriquto on Monday October 14, 2019 @12:47AM (#21245323)
But it almost is, already! Half of the lines are literally shell commands. And except maybe for the legal check, the rest of the steps can also be written as shell one-liners. For example, to update the version on a README file you can "sed -i" with an appropriate replacement, and so on.
Re:How to fuck up software releases by vortico on Monday October 14, 2019 @01:02AM (#21245391)
Can you explain how your ./deploy.sh script forks onto three laptops (Windows/Mac/Linux) and "plays around" with the installers/build a bit to make sure it works? I obviously can't release builds that don't even run because I didn't take a few minutes to test them.
Re:How to fuck up software releases by cobbzilla on Monday October 14, 2019 @08:47AM (#21247629)
Sounds like you are mixing release & QA, makes a lot of sense in a small shop.

In a larger team you’d have a small agent on each of your N (likely >> 3) machines, CI pushes to the agent for build/release/automated tests.

Then if those pass on a given system, it fires off a message to start manual QA validation for performance and other “intangibles”.

Re:How to fuck up software releases by enriquto on Monday October 14, 2019 @01:40AM (#21245527)
Why don't you use virtual machines?
Re:How to fuck up software releases by vortico on Monday October 14, 2019 @01:59AM (#21245594)
Good question. 1) A MacOS virtual machine is probably illegal. 2) I need to test the builds on all three machines, and VMs don't handle OpenGL, USB drivers (e.g. for MIDI controllers), and audio drivers very well or at all. 3) I need to have a rough idea of performance of the build. VMs affect performance by non-constant/unpredictable factors. For example, CPU performance might be 1x the speed of a bare-metal OS, but graphics performance might be 0.25x. 4) I don't know how to write a shell script on a host computer that launches Windows 10 and runs commands with an MSYS2 Mingw64 shell.
Re:How to fuck up software releases by enriquto on Monday October 14, 2019 @03:04AM (#21245834)
> A MacOS virtual machine is probably illegal.

true. And that is a very sad state of affairs. I use the travis osx hosts for that, but it's not ideal. There's no interface, but at least you can check that the code compiles, runs, and passes automated tests. That's already huge!

For linux and windows hosts, it seems to me that it is a solved problem, as pointed elsewhere.

Re:How to fuck up software releases by jakobegger on Monday October 14, 2019 @04:19AM (#21246131)
> A MacOS virtual machine is probably illegal.

It's not, as long as you run it on genuine Apple hardware. We use a bunch of macOS VMs that are 100% legal.

The problem with macOS VMs is that there are a lot of compatibility issues, and in my experience it's a lot of effort to set up macOS VMs. If you have the space and the money, real machines are a lot easier to deal with.

Re:How to fuck up software releases by mr_mitm on Monday October 14, 2019 @02:22AM (#21245668)
> I don't know how to write a shell script on a host computer that launches Windows 10 and runs commands with an MSYS2 Mingw64 shell.

Launching a VM via shell script is trivial. And you can install OpenSSH on Windows 10: https://github.com/PowerShell/Win32-OpenSSH

Re:How to fuck up software releases by jakobegger on Monday October 14, 2019 @04:14AM (#21246114)
We use a combination of real machines and VMs for building software, and dealing with macOS VMs is a major hassle. There are always compatibility issues that are hard to debug (eg. don't try to configure your VM to use more than 2 vCPUs if you want to run macOS 10.12). There is a lot of manual work that goes into maintaining VMs, and often automating those tasks doesn't pay off. If your app has any GUI components, those are often buggy in VMs (anything using the GPU is extremely unreliable).

Our main build machine is an actual Mac, because it's just so much easier to keep it running.

VMs are nice if you need a lot of different setups (eg. one app we distribute has components that need to be built on different versions of macOS, and VMs are nice for that).

Re:How to fuck up software releases by happytoexplain on Monday October 14, 2019 @12:41AM (#21245309)
How have you come to that conclusion?
Re:How to fuck up software releases by anon9001 on Monday October 14, 2019 @03:48AM (#21246026)
I can do every step on your list with any CI platform. I do something extremely similar to build apps for ios (macos), android (linux), and web (linux). I think it's so important that it's one of the first things I do on any project.

I wanted to give you an example, but it all seems so easy, I'm not sure where you're getting stuck.

Re:How to fuck up software releases by gdfasfklshg4 on Monday October 14, 2019 @05:37AM (#21246462)
Can you really do a legal review from a CI platform?
Re:How to fuck up software releases by wwright on Monday October 14, 2019 @06:52AM (#21246826)
If it can be automated, then obviously, yeah.

If not fully automated, many CI platforms allow you to pause the pipeline by requiring manual confirmation from an authorized user. I know Jenkins and CircleCI support this off the top of my head. You could have the CI perform any relevant searches or diffs, then display that to the user to get manual confirmation. It’s still not “perfect”, but it does allow you to reliably get someone to look at the data and say “yes” or “no.”

Re:How to fuck up software releases by anon9001 on Monday October 14, 2019 @11:41AM (#21249487)
Yeah, that's one approach. Another might be to commit some kind of `./legal/sign-off-v1.0.0` so you can see who did it, and so travis can know if it should deploy the build or not