One notable example is how their NDB client library used to automatically handle memcache for you, but they got rid of that with Cloud NDB Library and forced clients to implement their own caching.
The sequence of datastore APIs I've seen during my experience with AppEngine is:
* Python DB Client Library for Datastore[1], deprecated in favor of...
* Python NDB Client Library[2], deprecated in favor of...
* Cloud NDB Library[3], still supported, but they ominously warn new apps to use...
* Datastore mode client library[4]
[0] https://steve-yegge.medium.com/dear-google-cloud-your-deprec...
[1] https://cloud.google.com/appengine/docs/standard/python/data...
[2] https://cloud.google.com/appengine/docs/standard/python/ndb
[3] https://cloud.google.com/appengine/docs/standard/python/migr...
[4] https://cloud.google.com/datastore/docs/reference/libraries
It is hard to replace GCP's managed datastores because I really don't want to maintain my own database server (even if it's a managed service that someone else upgrades for me). So I've stuck to Google Cloud Datastore / Firestore, but I've been experimenting a lot with Litestream[0], and I think that might be my go-to choice in the future instead of proprietary managed datastores.
Litestream continuously streams data from a SQLite database to an S3 backend. It means that you can design your app to use SQLite and then sync the database to any S3 provider. I designed a simple pastebin clone on top of Litestream, and I use it in production for my open source KVM over IP. It's worked great so far, though I'm admittedly putting a pretty gentle workload on it (a handful of requests per day).
Having worked with quite a few ex-Googlers this is a pretty standard Google engineering pattern.
True, I have to maintain state on S3, but there's not much work involved in that.
If I was maintaining my own database server, I have to manage upgrades, backups, and the complexity of running an additional server. With Litestream, I don't have to manage upgrades because nothing bad happens if I don't upgrade, whereas there are security risks running an unpatched MySQL/Postgres server in production. Litestream has built-in snapshots and can replicate to multiple S3 backends, so I'm not too worried about backups. And there's no server to maintain.
What operational complexity do you see in Litestream?
But it also introduces a few challenges. It's not as easy to connect to your database remotely to inspect it, with something like SequelPro for MySQL. It's not possible to create an index or drop a column without blocking all writes, which can be annoying if your database is large. Database migrations in general are harder with SQLite because ALTER TABLE is limited. [1]
One last thing regarding losing the few seconds of data. If you use something like Google Cloud Regional Persistent Disk, then your data are replicated synchronously in two different data centers, which means you can lose your server, restart another one, and not lose any data. Can still be combined with Litestream for backup to S3 with point-in-time restores.
I don't see any operational complexity with Litestream.io. I think that's an awesome tool. But it's not that different of managing PostgreSQL backups with something like WAL-E.
The complexity of managing your own database server only exists if you don't use a managed service. Then there is no server to maintain and they do all the things you mentioned for you.
> If you're not already familiar with these tools consider using a managed platform first, for example Render or DigitalOcean's App Platform (not affiliated, just heard great things about both). They will help you focus on your product, and still gain many of the benefits I talk about here.
And:
> I use Kubernetes on AWS, but don’t fall into the trap of thinking you need this. I learned these tools over several years mentored by a very patient team. I'm productive because this is what I know best, and I can focus on shipping stuff instead. Your mileage may vary.
I actually spend very little time on infrastructure after the initial setup (a week of part time work, since then a couple of hours per month tops).
For comparison, this post describing what I did took nearly a month of on-and-off work. But I might just be slow at writing :)
I just wanted to complement your sentiment.
Real vendor lock-in is when you have decades of code written against an Oracle DB and you're getting charged outrageous Oracle rates and it would also cost a fortune migrate.
Real cloud vendor lock-in is when you have decades of code written against a [cloud vendor] and you're getting charged outrageous [cloud] rates and it would also cost a fortune migrate.[sic]
One thing I like to talk about to Cs is their strategy on capex vs opex, because honestly that determines quite a lot, but is often something engineers dont think about.
Not to mention “resume driven development”. Recruiters love cloud experience.
For example?
And in the end, it really depends on how much you trust different vendors and how you want to manage cash flows. Racking your own servers reduces some risks (Google deciding to terminate your account on a whim, Azure pushing wild updates, Amazon jacking prices wildly) while increasing other risks (only your own staff are watching your hardware).
Spend your early time working on your core business. If your core business isn't cloud agnosticism then you shouldn't be investing your resources there.
If you use something like AppEngine to run a Flask or Django app, you will not be locked in much because those are open source libraries with well known runtime options elsewhere.
Same to some extent with any sort of managed OSS database.
If you use something like Cloud Datastore or Firestore or DynamoDB , you are using a proprietary API and will have to rewrite all your client calls , or write an extensive shim, and probably significantly re architect to port.
Even in the “hosted OSS” option there are usually some vendor specific stuff but it can vary a lot. Something like AppEngine specifically used to be an absurd amount of API lock-in but has evolved over the years to be more of a general container runtime.
If all database access is compartmentalized and the two datastores are fairly similar then it can be pretty cheap. If you didn't compartmentalize it will be expensive. If their characteristics are different enough then your compartmentalization will probably fall down in some cases and it will probably be expensive, although not as expensive if it weren't compartmentalized.
The real heavy lifting would be if you've optimized your tables to that specific architecture. You might need to re-design a lot of your schemas.
1. Firebase Hosting for the React frontend
2. GraphJin (Automatic GraphQL to SQL Engine) on App Engine for the backend
3. Cloud SQL Postgres for DB
Heroku is great for general applications, but if you're trying to do something that isn't a standard CRUD app, it can really start to bite you in the arse.
Their DB pricing in particular is incredibly inflexible compared to AWS RDS. Among other issues we had with Heroku at my old job, was having a DB that was hitting its storage limits, but was miles away from hitting its memory or connection limits. There was no option but to upgrade to the next tier, with additional memory etc., even though all we needed was additional disk.
That's not to say that Heroku is bad, but like any tool, you need to be aware of the long term costs that are often associated with term convenience.
I run a NodeJS GraphQL server in App Engine Flexible, and it is basically just like running it in a Docker container. It's also pretty trivial to run in Google Cloud Run if I so desired, there is even a tool to assist: https://github.com/GoogleCloudPlatform/app-engine-cloud-run-...
Basically, your code shouldn't really need to change at all, it's really just your deployment scripts and configs that need to be updated. At their heart flexible GAE and cloud run are both just running Docker containers.
That said, using a static frontend cached on a CDN in general improves initial pageload and cuts down on traffic to your server by a lot. Netlify makes this easy if you want to use React on the client (with NextJS).
With AppEngine you get direct access in one console to all the bells and whistles of Google Cloud, basically the same as the other infra giants. AWS has even more bells and whistles but I find its console more annoying.
Heroku is similar, but Netlify is at least equally simple I found. Maybe someone else can shed more light on differences.
I've also paid for extra builds once or twice in the past (automatically charges a few dollars when you cross the build time limit), and I pay them $9/mo for analytics.
- Break down page views into unique visiters for all views (per site, per country etc.). (or some other comparison between those).
- Don't lose the history after 30 days.
- Export to .xlsx
There is actually a standard for this. I'm fine with MM, it's confusing to me but not ambiguous, just please don't reassign existing prefixes...
MM means millions, plural of M which means million.
The actual Roman version of a million is an M with a bar over it, where the bar means x1000. But that's not an ordinary character, so wouldn't work for this purpose.
I also have a high(er)-traffic frontend on a CDN which is used by their customers. User writes there are purchases/payments handled by third(fourth?)-party SaaS.
My current employers still haven’t learned this lesson and think caching fixes everything.
In a read-mostly environment you can often more easily afford to update the state all at once. It’s clear what the effects are because they happen sequentially. The cost of an update isn’t fanned out and obscured across the codebase, where or your team can delude yourself of the true system cost of a suspect feature.
As for shared mutable state, yes, that's true, but what are the alternatives? Whether it's memcached or Redis or an in-process cache (like Erlang/Elixir have), the tradeoffs seem mostly the same.
Caches are addictive. The first one is 'free' (easy) and people start wanting to use that solution for all their problems, especially social problems (we can't convince team A to get their average response time to match our SLA, so we'll just cache them to 'fix' it)
They defer thinking about architectural problems until later, when they are so opaque that "nobody could blame you" for having trouble sorting them out. But I do. Blame them, that is.
We're all in on AWS and don't care about lock-in.
The vendor lock-in argument isn't worth considering for most businesses.
Amazon will bend over backwards to accomodate a company spending $500 mil a year on hosting (apparently what Snap spends). Sure it's only a fraction of its revenue ($386 bill for AWS), but half a billion is half a billion.
What product is this?
https://lab.cccb.org/en/arthur-c-clarke-any-sufficiently-adv...
- Static frontend hosted on Netlify (free unlimited scale)
- Backend server on Google App Engine (connecting to Gcloud storage and managed DB via magic)
I realize I'm opening myself up to vendor lock-in and increased costs down the road (if I even get that far), but I've wrangled enough Docker/k8s/Ingress setups in the past to know it's just not worth the time and effort for a non-master.