I'm totally on board with the idea of being scrappy and taking shortcuts in order to get to PMF as soon as possible. However, it seems the proof is in the pudding here. If you can't service customers due to lack of compute resources, you can't get to PMF.
Also, yes there are certain infrastructure and network topologies that would absolutely be overkill for a young startup. I don't think multi-region is one of those things. I don't have experience with Azure directly, but on every other cloud providers, going multi-region is not something that requires huge amounts of time or resources. You just need to be mindful of it from the outset. And if you decide not to be, then at least be intentional and conscious about the risk and have a plan in place for what happens when you get bit by deciding not to go multi-region.
*privacy invasive that GDPR is explicitly set up to make harder, so duh
Also, lots of companies assert GDPR compliance via magical thinking. They most often are wholly wrong. Shopify can say whatever they want, but there’s no certification body.
Source: I’m the person who evaluates and builds compliance systems for a range of services you almost definitely use.
Sounds like customers are coming in thick and fast.
If this is the dynamic and the company can't spare a few weeks to solve it, something has gone seriously wrong in a very interesting way.
Without knowing the details about your services and infrastructure, it's hard for me to know what's involved in going multi-region now. Are you sure it's such a a gargantuan effort? I would've thought one person working full-time on this for a week or two would be enough, but again I don't know the details of your setup.
One option would be to pay a consultant who is an expert in Azure/cloud stuff to come in and help. May not be cheap, but could be a lot better and quicker for you and better for the business, especially if none of you are really big experts in Azure.
I've been here before (I think)...had to wear many hats and scramble to make sales, build the tech, act as de facto DevOps person even without a lot of experience doing it, etc. That is the way, but stuff happens.
Happy to chat about specifics if you want to bounce ideas off of me or go through your particular situation. Can't promise I'll have concrete advice, but happy to talk it through.
Very poor position to be in, apparently this happened in azure UK recently too.
Let the scar you get from this is be a learning experience, hopefully you will not fall into the same trap again to trust this company.
In my career I'm in a place where anyone suggesting I do work on Azure gets an instant doubling of my asking day-rate and I really hope the will be put off and find another victim for this gig.
That said, another learning experience would be to use terraform or something (tbh for azure the only sane thing is terraform, ARM templates are just garbage). Having terraformed your one region switching to the other would be much easier, tho not trivial.
If you rely on Kubernetes for orchestration and have minimal cloud API dependency, it may be worth that evaluating this option.
Also, do you have a TAM associated with your account? Are you just going through regular support channels? Can they deliver different instance types (not sure what the Azure parallel is), can they deliver short term capacity, etc?
I would try to push Microsoft more here. It's not like they've stopped on-boarding new customers into that region right? What happens if you create a new account in that region?
If you don't you're at a big disadvantage.
Similarly, just keep trying to change the size. Often it’ll go through when someone else decommissions something.
This is doubly worthwhile as if this stumble kills the startup (it can happen) this will be excellent experience to take to the next employer :)
The way to get more from most cloud is by becoming a partner, not just a customer. And the way to do that is increase dependency and usage.
(As an aside I also agree that multi cloud from the get go is a YAGNI violation. Just keep in the back of your mind “could we have an alternative to this?” when using your provider’s proprietary features.)
Just having the plan is already expensive enough.
None are ideal.
This is how the iphone was able to nuke windows phones which were designed to meet the needs of IT
Those MS problems have an “IBM mainframe” problem. New businesses won’t choose it.
That’s why I say “why do I want them?” If I was starting a new business I’d have no reason to use them.
AD = identity, access, privileging, SSO
In my mind they’re legacy business products.
Sure, most businesses use them, but I don’t necessarily believe that is a forever thing. At one point most businesses had mainframes.
I'm going to guess that the 5% that don't use Azure/Office 365 represent newer entrants to the Fortune 500.
71% of Fortune 500 companies use mainframes [3], and yet they are considered a dead technology with essentially no future. Do you know anyone or anyone who knows anyone who learned how to develop on mainframes in college in the current millennium? It sure wasn't part of my CS curriculum!
Small businesses represent almost half of US economic activity [1] and represent 99.7% of firms with paid employees [2]
[1] https://advocacy.sba.gov/2019/01/30/small-businesses-generat...
[2] https://cdn.advocacy.sba.gov/wp-content/uploads/2021/12/0609...
[3] https://www.precisely.com/blog/mainframe/9-mainframe-statist...
Multi-cloud is really not a big deal. Main nuisance is billing differences, followed by slight variations in e.g. Terraform config.
Multi-cloud should only be for mission critical infrastructure. Very little infrastructure is mission critical. Most other use cases can be temporarily wallpapered over with an "Under maintenance" page unless there's a good reason otherwise.
Multi-cloud introduces more risk than it prevents. Which is why things like simulated failovers and BCP testing is constantly required.
I don't want to be snarky, but when large service providers like AWS have their own crossregion downtime because one snowflake of a service in us-east-1 is down, I kind of dismiss the virtue signaling of high resilient multi-(az/region/cloud) ever existing in practice.
If you can somehow have a separate database per region/cloud, sure, I can understand that, but if you have to shard your database across many clouds, I'd dread having to tame such a beast, especially within a startup.
So you're saying it's impossible to improve reliability from 97% to 99% because you can never make it to 100%.
Multi-AZ and multi-region add complexity and cost much more quickly than they add reliability.
Sometimes it is worth it. Sometimes it is not.
> setting up in a new region would be complicated for us.
Sounds to me like you've got a few weeks to get this working. Deprioritize all other work, get everyone working on this little DevOps/Infra project. You should've been multi-region from the outset, if not multi-cloud.
When using the public cloud, we do tend to take it all for granted and don't even think about the fact that physical hardware is required for our clusters and that, yes, they can run out.
Anyways, however hard getting another region set up may be, it seems you've no choice but to prioritize that work now. May also want to look into other cloud providers as well, depending on how practical or how overkill going multi-cloud may or may not be for your needs.
I wish you luck.