News for hackers. Stuff that matters.
Re:An update on recent service disruptions by prepend on Wednesday March 23, 2022 @05:29PM (#30783665)
My fear is that this seems like a cover excuse for moving off MySQL. The bug will be too hard and they’ll move off. They will choose SQLServer and take a lot of time to convert and then have even more outages.
Re:An update on recent service disruptions by nimbius on Wednesday March 23, 2022 @05:53PM (#30783962)
100% agreed. a lift-and-shift migration to Galera and modern MariaDB wouldnt be hard, but knowing MS there are mid managers waiting in the wings to swoop in and drive this into the ground with azure/sqlserver, the former of which posted 8 outages in the past 90 days alone.

this is classic Microsoft. spend a ton of money for something very valuable -- in this case virtually all developer marketshare -- and then casually pedal it into the ground while you lie about the KPI's to C levels (IIS marketshare on netcraft as a function of parked websites at GoDaddy to dominate over Apache) and keep it on life support with other revenue streams (XBox) for the next 16 quarters until it becomes a repulsive enough carbuncle to shareholders that it gets the axe (Microsoft phone.) then in a year, limp into the barn with another product nobody else but you could afford to buy (minecraft) and slowly turn it into a KPI farm for Microsoft account metrics to drive some other failing product (Azure) and keep the C level happy while you alienate virtually every player with mechanics or requirements they hate.

Re:An update on recent service disruptions by prepend on Wednesday March 23, 2022 @06:15PM (#30784210)
It’s funny you mention Minecraft. My kids recently said “I hate Microsoft. All they did is ruin Mojang.”

They don’t know Microsoft for anything other than ruining Minecraft. They didn’t know Microsoft made the Xbox or even Windows.

They made this statement after Microsoft forced them to migrate their account they’ve had for 5 years to a Microsoft account. That broke their computer for a few days and reset their games. For no useful reason.

Re:An update on recent service disruptions by wyclif on Wednesday March 23, 2022 @11:26PM (#30786111)
My son is eleven years old, a huge Minecraft fan, and is exactly like that. Now, he does know that MSFT makes Windows, but that's not an existential threat to him like ruining Minecraft is.
Re:An update on recent service disruptions by illustrateth on Thursday March 24, 2022 @02:05PM (#30793421)
yes! i signed up for an account here just so i could say this. we got refurbished computers for my children just to play Minecraft. Minecraft didnt work in windows, so i set up a dual boot Ubuntu system (while biting my nails i might totally destroy the os), thatnwas working well for months. when they had to migrate their account, somehow it switched my daughter's local login to a Microsoft family login which screwed up her other apps (like adventure academy). im tearing my hair out trying to figure out how to switch it back to a local login. i already have two other apps for monitoring computer time (which already don't always work out as well as youd think, so i didnt want to really have to deal with a third one.) i guess ill eventually figure it out, but found this page looking for a solution. (and im not that techy right now, so it takes a good chunk of time.... sigh
Re:An update on recent service disruptions by deckard1 on Thursday March 24, 2022 @01:12AM (#30786559)
wait until Microsoft starts screwing up Blizzard IP. Or, I guess, screwing it up more than Activision-Blizzard already did...
Re:An update on recent service disruptions by throwusawayus on Wednesday March 23, 2022 @06:02PM (#30784062)
galera is not a solution for scaling out writes, full stop

galera has lower max writes/sec than a traditional async single master because it's a cluster. the other members of the cluster need to ack the writes, and all members are doing all the writes, so adding machines does not increase your max writes

Re:An update on recent service disruptions by andrew_ on Wednesday March 23, 2022 @09:26PM (#30785542)
Looking for education here: in reading my understanding of MairaDB was that it's not as well suited to large databases and heavy read/write performance in those databases, as compared with Postgres. Would the primary argument be for MariaDB's sharding support first-class?
Re:An update on recent service disruptions by drewbug01 on Wednesday March 23, 2022 @06:06PM (#30784104)
At GitHub’s scale, you don’t just “move off” a database. At best it would be a gradual project that would take years for the company to complete, and likely trigger additional incidents along the way.
Re:An update on recent service disruptions by samlambert on Wednesday March 23, 2022 @06:55PM (#30784625)
Correct.
Re:An update on recent service disruptions by protomyth on Wednesday March 23, 2022 @06:16PM (#30784225)
Why would Microsoft not migrate to SQL Server? MySQL is owned by Oracle. Microsoft cannot be happy about using a product from Oracle. SQL Server is a pretty good product and the conversion will give them even more tools and expertise for their consulting wing to do it for other companies.
Re:An update on recent service disruptions by macintux on Wednesday March 23, 2022 @08:35PM (#30785267)
They famously tried to migrate Hotmail to Windows after purchasing it, and that proved a long, excruciating process.

I doubt they have enough to prove wrt SQL Server to make it worth going through that again.

Re:An update on recent service disruptions by Ygg2 on Wednesday March 23, 2022 @06:51PM (#30784594)
They do seem to be acting like they are cool and OSS friendly.
Re:An update on recent service disruptions by protomyth on Wednesday March 23, 2022 @07:35PM (#30784938)
OSS friendly is one thing, but not using one of your flagship products is another. Plus, MySQL is Oracle and who knows where that goes.
Re:An update on recent service disruptions by prepend on Wednesday March 23, 2022 @09:43PM (#30785621)
Microsoft could easily fork and maintain mysql if oracle did anything wonky. And they could use MySQL without paying any licensing fees due to its OSS license.
Re:An update on recent service disruptions by qbasic_forever on Wednesday March 23, 2022 @07:28PM (#30784869)
Does it matter? Github is a closed source service. If they run Microsoft SQL Server vs. MySQL it doesn't matter or change anything for its users. All that us as customers and users care about is the experience and performance. Whatever they want to do internally to achieve that doesn't matter at all IMHO.
Re:An update on recent service disruptions by prepend on Wednesday March 23, 2022 @09:42PM (#30785617)
It matters because it makes their product worse. I’m a customer so I’m impacted by using a crappier database that has higher costs and worse performance.
Re:An update on recent service disruptions by baq on Thursday March 24, 2022 @01:18AM (#30786576)
SQL Server is at least as good as any variant of MySQL and that’s only if you want to be very generous for MySQL.

If you meant that and I misunderstood, the other issue is that you don’t migrate to a different RDBMS. You rewrite half of the app and then spend a couple years fixing issues.

Re:An update on recent service disruptions by stillblue on Thursday March 24, 2022 @12:43AM (#30786443)
SQL Server is crappier? How?
Re:An update on recent service disruptions by nodesocket on Wednesday March 23, 2022 @08:38PM (#30785278)
Microsoft SQL actually is pretty good and really performant (with my limited production use).

I would argue it’s more performant than vanilla MySQL and it supports multiple write masters using peer to peer transactional replication an enterprise license feature (https://docs.microsoft.com/en-us/sql/relational-databases/re...)

Re:An update on recent service disruptions by prepend on Wednesday March 23, 2022 @09:45PM (#30785632)
It’s not a horrible database, but it’s just expensive and pretty much never a better choice than Postgres or other OSS dbs.

There are certainly some very niche use cases where you need it, but there’s a reason why tech stacks don’t use sqlserver. I think there are better ways to handle durable transactions and redundancy than using sql enterprise with sql‘s replication.

Re:An update on recent service disruptions by thecosmicfrog on Wednesday March 23, 2022 @08:30PM (#30785246)
Counterpoint - with GitHub now being part of a large database and cloud vendor (i.e. SQL Server and Azure), surely there are at least some merits to moving workloads to those technologies? Leverage internal knowledge, talent, etc.
Re:An update on recent service disruptions by jiggawatts on Thursday March 24, 2022 @01:39AM (#30786657)
"talent"

Nothing I have seen about Azure SQL fills me with confidence about their technical capabilities.

Performance at all tiers is woeful, the networking connectivity is madness, and disaster recovery doesn't.

Re:An update on recent service disruptions by ankraft on Thursday March 24, 2022 @04:27AM (#30787395)
I’m curious why you think this way, can you elaborate?
Re:An update on recent service disruptions by jiggawatts on Thursday March 24, 2022 @06:20AM (#30787979)
Don't take my word for it. Spin up an Azure SQL database and try it yourself.

"Ping" it using a trivial query such as "SELECT 1" a thousand times in a row and draw a histogram. Or just eyeball the numbers. I did this recently and had response times over 12 milliseconds regularly. For comparison, a small and cheap IaaS VM running SQL Server can get down to the 150 microsecond range and stay there.

For this 100x degradation in performance you get the privilege of paying several times the cost of an IaaS VM + SQL license.

Azure SQL proxies connections. I mean sure, the documentation says that they can do a "redirect" instead of a proxy, but not if you use any of their "private" network connection options. You would think that is some sort of simple IP header change implemented by the switching gear in hardware, but you'd be wrong -- it tunnels through what is essentially a VPN -- with all of the predictable issues. These tunnel VMs don't have accelerated networking turned on, for example. So you can have "business critical" tier on one end, and huge VMs with accelerated networking on the other end, but the traffic in between is being routed through some 1 vCPU virtual router appliance processing packets in software.

Anyone with database admin rights can alter firewall rules via SQL commands. These are via SQL Server accounts and hence they're just a username and password, no MFA or anything. If you can find a SQL injection vulnerability you can punch a hole through the "firewall". Brilliant.

The firewall supports IPv4 only, and uses different CIDR syntax to everything else in Azure. It doesn't support Service tags, or logging, or monitoring, or anything really. It exists only to tick a checkbox.

Unlike most other Azure services, SQL doesn't integrate with Azure Active Directory or RBAC properly. So for example you can have one AAD group as the SQL Admin. No other rights, no list of principal IDs... just one admin group. All other delegated permissions must be done through SQL commands, blocking the use of ARM templates, Policy, custom roles, etc...

If you delete an Azure SQL Server instance, it deletes all backups associated with it. Sure, they recommend that you put a "delete lock" on the server, but then most administrative operations become impossible because you can't then delete any child objects. And even if you do create a delete lock, that can just be deleted. There is no way to say "backups cannot be deleted for at least 'x' days", even though the underlying storage accounts support this feature.

DISCLAIMER: Some of the above may have changed since I last checked, always read the documentation and/or verify with support if your data matters to you.

Re:An update on recent service disruptions by grnmamba on Thursday March 24, 2022 @09:44AM (#30789641)
> If you delete an Azure SQL Server instance, it deletes all backups associated with it.

That... is a massive footgun. Essentially a 1-click method to destroy your business.

I'll set up delete locks ASAP, thanks for the tip.

Re:An update on recent service disruptions by speedgoose on Wednesday March 23, 2022 @05:33PM (#30783712)
I bet on Azure Cosmos DB.
Re:An update on recent service disruptions by antn on Wednesday March 23, 2022 @10:02PM (#30785715)
Nope.