Spanner doesn't exactly hide the details, but it lets you make transactions that span multiple shards. You still eat the cost of the transaction, you're just free from having to implement it at the application level, which is a more difficult and error-prone way of doing things. The bottom line is that if you need consistency, it needs to be implemented somewhere in your stack. If you don't need consistency (analytics workloads come to mind) then you have more flexibility with your database.
Disclosure: Google employee, reconstructing what I know from published information.
Discolsure: Also a Google employee, also reconstructing.
Google: Make unique products that push the boundaries of what was previously thought possible.
Amazon: Don't care about inefficiencies and usage. Inefficiencies can be handled by charging more to the clients, usage doesn't matter because the users are mostly the clients and they don't feel their pain.
Google: Had to make all their core technologies efficient, performant, scalable and maintainable or they couldn't sustain their business.
Amazon: IaaS
Google: PaaS
Amazon is philosophy is being 'close to the metal' to allow Enterprise customer to migrate 'regular apps' into a 'regular environment' in the cloud.
Most of Google's offerings are (at least were) novel, but proprietary ways of doing specific things.
Amazon is not a laggard: they have provided a number of interesting and useful 'helper' things to facilitate IaaS - as well as a number of 'pure cloud' type things.
Amazon is very, very customer focused. Their products come from customer demands.
Google often 'cool things they've done internally' and exposes them, hoping that they might have some use-case in the rest of the world.
Google and Amazon are equally interested in profit.
Which Amazon totally didn't have to do with their firehose of cash?
Amazon runs nothing, it's an outsourcing firm. They needed to make services "good enough" to be sold. If a service is somewhat inefficient, it just charges the clients more to cover the costs.
Technologies reflect the business they were created in.
What the fuck are you talking about. It's one thing to say AWS services are "good enough" but "Amazon runs nothing" is a ridiculous statement.
I suspect that Google knows this, and their reputation for have poor customer support and sales comes from that knowledge.
AWS prioritizes building blocks that support very high throughput and avoid leaky abstractions at all costs, and they're happy to push forward as long as these criteria are met. IMO they really succeed at this goal. Minus specific bugs that they're generally good about acknowledging, their services reliably do what they say they're going to. And they definitely solve a lot of problems for you, even if sometimes you're still required to get further into the weeds than you might want.
I'll buy that Google Cloud is better at questioning underlying assumptions and sometimes succeeds in releasing higher-level abstractions than AWS without any leakiness (a great example of this now being Spanner vs Aurora). It also feels to me that with releases like this Google is leveraging the full value of their own experiences running their services, and seems to be more advanced than amazon in some areas so this has a lot of value, whereas AWS seems to build a broader range of products with a specific customer in mind which is not necessarily themselves (e.g. all of their move your on-prem stuff to the cloud helpers).
If you consider Spanner vs Dynamo, it definitely matches up as Google wrapping the old way and Amazon forcing a new way (though to be fair, Dynamo was released 5 years earlier). But on the other hand considering Spanner vs Aurora, Amazon is the one embracing the old way with full MySQL & Postgres compatibility whereas Spanner sounds like a pretty dramatically different subset of SQL in not supporting insert and update statements. It's a very reasonable compromise for basically getting to ignore the CAP theorem, but it is a a significant difference that every developer will have to learn.
(work on google Cloud)
Google tried that a decade ago and found it lacking, this is why Spanner exists in the first place.
You're both right.
I don't think it's easy to port existing applications to use it and in the end you will still need to accommodate shortcomings in your application.
Either way, they are trying to abstract away having to think about eventual consistency with this offering.
The thing thats really different here is Google are basically saying, heres this awesome system, yes it has obvious risks from partitioning, we are going to stake our reputation on those partitions not happening.
In contrast AWS are saying, this is DynamoDb, its really limiting but because of those limitations it should be pretty reliable as long as you write your application correctly.
It will be interesting to see if Microsoft and Amazon have to follow Google's lead here.
No, they said this in the F1 RDBMS or Spanner papers. They originally did the NoSQL, eventual-consistency type of stuff. This had app developers required to do a lot of work to avoid problems that model can create. Apparently, even their bright people had enough problems with it that they decided to eliminate or simplify that situation with stronger consistency. Took some brilliant engineering but now they have a database easy to use as old model with advantages of newer ones.
If anything, they learned some hard lessons with a good solution to them. Now, they're offering it to others. I was hoping they'd do this instead of keep it internal only. F1 and Spanner are amazing tech that could benefit many companies.
As cutesy of a sentiment as it is, it's also full of misconceptions. The pens were invented by an American corporation that wanted better pens to sell in general (a smoother flow in a pen, regardless of gravity/orientation, is a better pen), and they saw a good opportunity to market the pen to NASA for use in space. Both NASA and the Russians used pencils in space, but the problems with pencils is the flakes can pollute an environment pretty quickly in low gravity and the pens turned out to be a much better solution. (So far as I've heard, every space agency these days buys similar pens.)
I meant the differences in design philosophy that permeate aerospace engineering on both sides. Russian, built ugly but for strength and longevity. American, built for high capability with finesse and finer tolerances. The emergent properties of these different principles explain why Soyuz is still a preferred launch vehicle, but it was the Americans who got to the moon and operated the STS.
Amazon is more like the Russians: built in the knowledge that things fail, but less magical as a result. Google is more like the Americans: remarkable technology, you just need a herd of geniuses to run it.
It is an interesting analogy this dichotomy you see in the design philosophies (both between the space programs and the mega-corporations), but perhaps my point, if I were attempting a point, is to beware of false dichotomies.
However, by 'abstracting' this away, you're not being forced to think about failure domains. If there is ever a massive country-wide connectivity break to the wider Internet (feasible for lots of people inside censored countries), you'll be pretty pissed when you can't use the DB services for your servers in the Google-local datacenter that you still have connectivity to because it can't get quorum.
That's the motivation behind both Spanner and F1: take the experience of how painful it is to do transactions on a Regional or Global level, and never make individual teams do it again.
I see it a bit like "Don't roll your own crypto". Clearly some people are exempted from it, but you better be able to tell me why you get an exception.
Disclosure: I work on Google Cloud and want you to pay for Spanner :).
Google prefers building advanced systems that let you do things "the old way" but making them horizontally scalable.
Amazon prefers to acknowledge that network partitions exist and try to get you to do things "the new way" that deals with that failure case in the software instead of trying to hide it.
I'm not saying either system is better than the other, but doing it Google's way is certainly easier for Enterprises that want to make the move, and why Amazon is starting to break with tradition and release products that let you do things "the old way" while hiding the details in an abstraction.
I've always said that Google is technically better than AWS, but no one will ever know because they don't have a strong sales team to go and show people.
This release only solidifies that point.