I'm not really sure where you're coming from.
As someone who's been doing this for years, the old days when you had a £10 virtual machine that could handle tens of concurrent requests don't seem to be handled by the low price AWS/Azure servers. Let's not forget, tens of concurrent requests is 100,000s of monthly users, a major win for most startups.
And even the expensive cloud options are utterly rubbish compared to what you can get for a bare metal, I like to compare the two as you can get a Ferrari for £1000 a year, or you can get a bicycle for £1000 but don't worry, it's a "cloud" bicycle so definitely go with that.
I am constantly having problems with an Azure app at the moment timing out because of some poorly optimized queries that are being throttled because they somehow exceed the ever mysterious DTU. On my laptop, a 5 year old laptop, the queries take 10ms. So it's is impossible to run on Azure because "reasons" unless I pay a ridiculous amount of money to upgrade the SQL server. Yes, I could fix the query, but why should I be spending time on that? They're part of the admin system and are simple reports.
Another app I worked on we ended up having to pay £1000 a year just so we could have multiple-domains + SSL, when we expecting to pay £80 p/y.
Personally, every time I use cloud servers I regret it. I keep trying or clients really want it, and can understand the lower devops cost and that with bare metal, every 5-10 years I need to migrate the app to get a better server, but I honestly find the cloud servers so completely underpowered for the cost. I don't understand how they've got so popular. Basic devops for most startups is a day or two task once every few years. Write a deploy script, backup script once, and almost forget about it.
(I absolutely love s3 though for backups though)
1) bare metal is dramatically faster than most cloud instances. Many of us might have collection of old hardware that, by performance, would be equivalent to >$1,000 in monthly cloud fees.
2) If from the start of your career you have only dealt with modern cloud/Azure/AWS, there is a whole body of knowledge you are missing. Acquiring it from scratch is time consuming, and you are never quite sure you are doing the right thing. As a developer, I don't pay the AWS bill, so it would be more rational to spend my time on learning ML or something else that will be reflected in my paycheck.
Any advice would be greatly appreciated. Can’t handle the monthly payments to gcp only to get a raspberry pi level of compute.
I point my DNS at the VPS, then I use the Zerotier VPN to tunnel traffic to my home server (Zerotier is embarrassingly easy to set up - I struggled for a while with OpenVPN but gave up). I use iptables packet forwarding to route traffic on port 443 or 80 from the VPS over the VPN to my local server, which is running an Nginx instance to serve the traffic.
This works perfectly for me 99% of the time. Main pain points have been the unreliability of hosting a server locally where Internet can die and power drop off, and a small amount of sysadmining on the VPS, which is so tiny that I have to be mindful of disk space (e.g. removing old Linux upgrade images every so often, etc). All told though, it works surprisingly seamlessly, and there's a certain smugness to knowing traffic from my smart TV goes all the way round the world and back through a VPN just to talk to my server sitting one foot below the TV.
I have it easy for external access -> static IP and a Gigabit connection without NAT, so all I have to do is setup ports on my router.
For my services like databases and Redit, I expose them only on Cluster-IP, and them you use kubeDNS and get something like 'redis.default.svc.cluster.local'.
If I need to access them for development purposes, I use kubectl-proxy, but a VPN is also an option.
Edit: I think I misunderstood, but will leave this up anyway.
I am also thinking along these lines but not sure about reasonable paths to try. Care to share some pointers on your setup? Basically looking to serve web pages from a server cluster at home.
If you look at Microsoft's strategy with C#/Azure you'll see that every single tutorial uses deploying on Azure. Their instructions for how to deploy to a real server in their tutorial is pretty much "we're not telling you, figure it out yourself, sucker".
Their new business strategy is to get the entire of their userbase paying massively over the odds for hosting small apps.
And, in reality, almost every app by any startup you could go and look at on producthunt (and probably most of ycombinator startups) have such low volumes of usage they could run on a $5 VPS. But I bet all of them are on AWS (there's even a few comments on this thread that basically say "start on AWS, then move").
Everyone is using cloud now, go to any meetup and ask. Or simply look at the sales figures.
I really like that for my side projects/solo hacks I don't have to think about anything other "here is my code that does the thing that I think is cool" and all the deployment/serving/data layer/ml model inference/etc just kind of happens.
For small/very early startups it's also similar. At least in the US, it's not the extra $5k/year you spend on cloud hosting vs doing it yourself that kills you; for many engineers that less than a week of comp.
At the same time, when you are large and spending $100s of k on your hosting then sure, look into alternatives and migrate if needed. This is where "start on AWS, then move" comes from.
I can make a script to do it all for me (and have in the past), but it's a bit of work.
That's really why the title of the article is apt being "the Amazon Premium", and not "the Amazon Rip-off".
The other side of this is that documentation is hard, and Microsoft documentation is frequently lacking or out of date, especially for anything off the beaten path. I use IoT Hub for messaging, and it's documentation is scant, many limitations and failure modes are found by trial and error.
Magically absorb the knowledge in an incantation to the Redmond Gods in the ether?
On reflection my comment about level of skill is unfair, what I should have said is: generally the information is there, but it's not always in the form of follow-able 'How-to' guides, often it's a dry detailed documentation. If you need something other than the standard setup, it is up to you to know what you are looking for and put 2 and 2 together. Of course it is by no means easy.
They do have reasonable guidance on how to deploy ASP.net core on Linux, but it can be lacking for lesser used stuff
Ironically, we were having the discussion here yesterday about a guy with a failed startup saying he'd wasted the opportunity because he used too much new stuff and spent all his time on that instead of iterating the actual product. Several comments focused on it'd just be better to start off with a Rails app, it's quick, it's easy and it's understood.
And really, is lambda appropriate for anything more than a very focused niche app? I mean, try writing all the usual things you need to slot into a SaaS on it, billing, user management, signup, admin reports, admin functionality, emails, multi-tenancy, custom domains, etc. etc. Even a 20 user, internal, enterprise app needs most of that functionality. Put on top of that, you're forced to use a particular, proprietary tech stack and lock yourself into something that might very well die in a couple of years.
Lambdas always free tier is 1 million requests per month and 400Gb/seconds per month.
https://aws.amazon.com/lambda/pricing/
It takes a lot to have any meaningful lambda bill. That in no way is meant to be an insult. If you’re spending even $100/month on lambda, you’re running a successful website.
It's hard enough starting a business without learning a whole new tech stack.
If you know how to write a program and create a function that takes two parameters - a payload and context argument - you know lambda. You can write it in the console if you’re using Python or Node and just run it from there.
Even if you don’t want to learn that, you can host your standard C#/WebAPI, Python/Django/Flask, Node/Express API on lambda just by adding code and changing your endpoint.
Several comments focused on it'd just be better to start off with a Rails app, it's quick, it's easy and it's understood.
There is also code to wrap a Rails app and run it on lambda....
And really, is lambda appropriate for anything more than a very focused niche app? I mean, try writing all the usual things you need to slot into a SaaS on it, billing, user management, signup, admin reports, admin functionality, emails, multi-tenancy, custom domains, etc. etc. Even a 20 user, internal, enterprise app needs most of that functionality
Yes. Especially in today’s world of client side frameworks and microservices, you host your static assets like Javascript, css, and HTML in S3 and call server side APIs hosted on lambda.
Put on top of that, you're forced to use a particular, proprietary tech stack and lock yourself into something that might very well die in a couple of years.
Well, two things. What’s more likely to die in two years / your startup or AWS?
Second, you can host your standard API just by adding four lines of code and you don’t have to make any changes to host it in Docker or on a VM.
There is a way to do this in every language that lambda natively supports. For instance for Node:
https://github.com/awslabs/aws-serverless-express
I have a Node/Express app that gets deployed to both lambda and Fargate (Serverless Docker) when I push it. The only difference is that lambda uses one entry point and Docker uses another.
I’ve done something similar with C#
https://aws.amazon.com/blogs/developer/deploy-an-existing-as...
Where it is deployed to both lambda and a Windows server running IIS.
I can be a lot more productive with all the resources that AWS has to offer when all I have to do is provision resources with a CloudFormation template and instead of doing the grunt work of infrastructure administration I can concentrate on writing software.
In the case of GCP, the product is the product rather than the user, so there's a greater expectation that it'll not get shut down or that pricing will change easily.
Further, they're obviously committed to being competitive with AWS which requires that they behave similarly.
Maps is a consumer facing service for which revenue is derived from the data collected and advertising. That business model is nothing like selling commodity compute services to enterprises.
If anything, I think GSuite is a portent of how Google will treat GCP. That is, the set of products has been relatively and consistently stable for quite some time, which has driven fairly decent adoption into the enterprise market.
There've a couple of product shifts, specifically around voice, hangout, and chat, but that seems to settled at this point.
I speculate that the changes that have raised the most ire have been around gmail, which I suspect has a fragmented product management team as it's both a free consumer service getting monetized through ad revenue as well as an enterprise product sold directly to customers.
So selling access to an API that businesses depend on is nothing like selling access to other resources that businesses depend on?
https://gadgets.ndtv.com/apps/features/google-maps-apis-new-...
Prices have gone up as much as 14 times in certain scenarios
Selling access to the Google maps API is wholly superfluous to Google's business model. Certainly, it doesn't clock anywhere near a billion in revenue.
Prices have gone up because Google doesn't have any interest in sustaining the product. The total addressable market is miniscule by comparison to cloud or even enterprise productivity apps.
It doesn’t matter how large the “addressable market” is if Google doesn’t have the will or talent to compete. Social is also a large market, how did that work out for Google?
Isn’t that the entire pitch of every startup? “If we just get 10% of the market, we will be huge”.
And “revenue” is meaningless is GCP profitable? There was a submission on the front page of HN yesterday about Google’s management not being happy about how GCP waS doing and they have three years to turn it around or “lose funding”.
At worst, they'll sell off or divest the business entirely. They'll expose themselves to tremendous liability if they attempt to wind it down like their other properties.
On the other hand, AWS announced something as minor as that they were discontinuing one form of accessing publicly available S3 urls that was deprecated in 2010 for the newer one. After a vocal outcry from the internet about how many websites that it would break, they backtracked:
https://aws.amazon.com/blogs/aws/amazon-s3-path-deprecation-...
On the otherhand, how many Urls were broken when Google discontinued it's link shorterning service?
Google has shown time and time again that it can't be relied upon for any important infrastructure.
Yes, it's true that Google capriciously kills products for various business reasons without any regard for customer sentiment. My counter-point to that is that GCP products exist under different conditions, specifically legal and business realities. They have contractual obligations to corporations with very large legal departments. There is also the business reality that if they're going to compete with AWS and Azure then to be competitive they have to behave in similar ways. If they piss on customer sentiment as they have with their b2c services, they'll fail and I think this is very clear to them.
That's not to say that they aren't already failing. As I mentioned up thread, their support is laughably bad and when pressed about that fact, they don't seem to actually acknowledge how that could be a problem.
Also the GCP panel is truly awful, much much worse than the AWS console.
However, their support is absolute shit and they're absolutely devoid of the customer first mentality that AWS demonstrates.
Doesn't matter what is the product. They're ready to kick GCP to the curb if it doesn't perform up to expectations. I wouldn't host anything important on it.
For some more difficult use cases I have to be pragmatic and decide on the circumstances. E.g. when needing presigned URLs (which is an S3 feature) I will use the native S3 SDK, it's not the cleanest way, but it gets the job done. Only when it starts getting annoying / slowing the development speed will I abstract it away. Security is important and I keep a high standard there, but when it comes to clean abstraction I will occasionally make compromises, knowing that often we will only need one implementation or that feature might be thrown away (I work freelance mostly with startups or SMEs).
It is in the ecosystem in the sense that it ultimate boots up an EC2 instance. It is essentially a simplified skin on top of EC2 where AWS makes most of the decisions for you in regards to server setup, with simplified pricing. And it shows up as AWS on your bill, and can be billed along side other AWS services you use.
However, it isn't really in the ecosystem as I would argue. You can't connect it to networks and subnets you create. You don't have control over availability zones, so essentially everything (like talking to S3) is considered egress traffic. It is like a sandboxed AWS service, that sits out there on its own. It really isn't any different than using Linode or Digitial Ocean for your compute resource and using AWS for block storage, queuing, DNS, etc.. Other AWS services can't "see" your Lightsail instance, like they can see your EC2 instances. You can't connect EBS volumes to it, you can't take advantage of elastic IPs, or anything of that sort. It really is a separate service that sits on its' own, under the comfort of the AWS name.
If I remember right, even tags don't carry over between Lightsail and the rest of your AWS account. Again, Lightsail is fine as a service, but it is only "sort of" in the AWS ecosystem. It mostly sits on its own as an isolated service, backed by AWS.
Being a "solo hacker" developing multiple websites, I need to prioritize where I put my time in. Thus my "default" choice is using Elastic Beanstalk to deploy websites, be it PHP, Python or something else. I can easily add environment variables, let it connect to RDS (database), scale it up (more servers or bigger servers), I can add SSL certificates, I can add a CDN (Cloudfront), I can add separate data storage (S3) all with a few clicks and from one (albeit super-huge and need-to-get-used-to) dashboard.
Alternatives: I don't trust Google services for anything critical (they tend to shut down easily or they might just change their pricing drastically - e.g. in Google Maps), Azure might be an alternative, just never had the time to check it out intensively.
For "fixed ceiling" projects, I use Digital Ocean (because unfortunately AWS mostly doesn't offer a budget ceiling, just alarms).