Turns out the uptime of both our git server and even the Jenkins instance beat GitHub by far and while the former only cost a marginal amount of CPU time on infrastructure I was running anyways, GitHub is a noticeable expense for us.

Of course it still saves me from the panic attacks every time I'm compelled to press the "Update now" button in Jenkins because either I do nothing and get my instance RCEd or I do press the button and who knows what plugin update will break which part of our setup, but while that was a constant fear in my mind, the amount of downtime caused by Jenkins plugin updates was zero whereas what GitHub is doing lately is way, way, way worse than zero.

I'm starting to get frustrated and like I presume many other paying users, I think I'm at a point where I feel like we should get partial refunds of our subscription money given the very spotty uptime all year now.

Never expose Jenkins to the public internet, make sure it's via VPN. If you need webhooks, there are services for that which allow you to broker webhooks whilst calling in from the Jenkins side (i.e. not exposing ports). Even so if you do have to use native webhooks, at least lock it down to the upstream's IP range(s).

Ideally have a dev jenkins to test all the things first before hitting upgrade on your prod instance and killing some plugins (hell even better if it's all IaaC and can just spin up a jenkins host per env, but ££$$$££/Time etc)

And GitHub's SLA is not great to begin with: 10% of spend refund at 3 nines (99,9%) and 25% of spend at 2 nines (99%).

How is it mediocre? Is it because of the CVEs that have been released in the prior years? I recall GitLab also having quite a bad week of CVEs in February[1].

How is it a bad ecosystem? If this is about plugins in order to do things, I actually like this framework - it lets there be specific owners for portions of the open source development.

Self-implodes? This seems like it would be tracked as a bug. I've encountered an instance where Jenkins wouldn't start due to a crypto issue but that was due to a bug and all I needed to do was install a patch.

I think that using Jenkins can be a thought of a serious option if like anything else, you follow security protocols ie: don't allow public access, maintain RBAC standards, have a maintenance schedule.

[1]https://about.gitlab.com/releases/2022/02/25/critical-securi...

The problem with dependencies is that different plugins need different versions but there is no solution to that. Even more. You update Jenkins. Plugins break. Data gets corrupted. You try to align the plugins to some dependent version. It fails, cause most of the ecosystem is a hobby-project that never gets updated.

Jenkins has absolutely nothing to do on the public Internet as most of these tools.

There are so many CI/CD services. Most are very cheap. You really need to love yak shaving to pick Jenkins over something from the shelf. No major cloud offers Jenkins Saas. Wonder why…

'I don't like it because it's not secure and I was able to get access to prod via this method.'

'That method has been fixed.'

'Well...I still don't like it.'

Bugs are being found and fixed all the time. As for a the different plugins need different versions. Yes. This also occurs in general coding to where every package or library that you want to use in your code is also going to be versioned. If that package or library gets updated, your code might break.

This goes back to having some things in place. Backups and a maintenance and testing schedule. If you don't test things before pushing them out to prod maybe fix that first.

[1] https://docs.gitlab.com/ee/ci/yaml/gitlab_ci_yaml.html

We've been running GitLab on GKE for the past three years, no problems outside of initial migration pains.

https://www.microsoft.com/en-ca/servicesagreement/upcoming.a...

So what part of the github terms gives them permission to sell my IP back to me through copilot?

GitHub has a separate policy...

As I asked elsewhere, where is the part of these agreements that permits them to profit off my IP through copilot?

If you're publishing your IP to GitHub in a public repo and using a widely permissive license, they didn't need an agreement to use that IP to train copilot.

That's what I want. To find that line.

So, where does it say they can use my IP?

Or I guess you didn't really have any IP of value that was infringed on whatever license you claim to have made.

Buf if you published your code with an MIT or GPL kind of license chances are they don't owe you anything nor do they have to say that it's using your code. IANAL, but nothing in those common licenses leads me to believe they owe you even that statement. If I'm wrong feel free to point to it.

You're not really engaging in this discussion much more than giving extremely vague hypotheticals and then making demands that seem like quite a leap. Where in, say the GPL, would they have to make a statement like you claim they need to? What are the actual terms in your public license and code that you have that they would have infringed in some manner? The repo would have to be public and you just said it would be free, so please do link it. Otherwise, what are we even talking about here? Just making noise on the internet I guess?

And how is it disparaging to suggest that if you really think Microsoft somehow harmed you that you go and do the thing that would actually resolve the issue?

I try and learn from my mistakes. Can you describe exactly how I am being disparaging by these comments?

Also, following up, will you share the codebase in question? You said its free, and its clearly something publicly hosted in GitHub otherwise its entirely out of scope from this conversation.

Oh you sweet summer child

For a second I thought someone must have deleted the actions yaml files.

This is a dangerous failure mode.

https://github.com/multiprocessio/dsq/pull/82

Screenshot here: https://twitter.com/phil_eaton/status/1542168020516216832

(I work at Microsoft, but not at Github)

Edit: It's just the HN title that says "now resolved." This github status says:

> We have identified the source of disruptions and are actively working on a mitigation. The systems are in recovery and services are returning to green.

Who shouldn't at the very least donate a bit to the various open source CI solutions, as a way to have some kind of hedge?

I'd recommend starting with one of the workshops listed here [0] and maybe dive into more learning resources and the documentation.

Our community platforms, such as the forum [2] and Stack Overflow, etc. [3] are also a great place to ask questions and collaborate on challenges.

[0] https://about.gitlab.com/handbook/marketing/community-relati...

[1] https://about.gitlab.com/learn/

[2] https://forum.gitlab.com/

[3] https://about.gitlab.com/community/

There's a point where it's funny, and we're way past that.

Each time this happens, it makes no sense to go all in on GitHub. Perhaps companies like ARM, and projects like Wine [1], ReactOS [2], etc already went with self-hosting or have a failsafe solution to fall back on.

[0] https://news.ycombinator.com/item?id=31815918

[1] https://www.phoronix.com/scan.php?page=news_item&px=Wine-Git...

[2] https://github.com/reactos/reactos#code-mirrors

[1] https://www.githubstatus.com/

Anyway to minimize the impact of such github incident on everyone's daily projects and business?

Jenkins is slow and a nightmare to maintain. It became a huge ball of mud that nobody wants to touch. Just keeping the lights on it's a large burden for the infrastructure team.