News for hackers. Stuff that matters.
Re:Introducing Cloud Spanner, a Global Database Service by jedberg on Tuesday February 14, 2017 @01:22PM (#13645755)
This release shows the different philosophies of Google vs Amazon in an interesting way.

Google prefers building advanced systems that let you do things "the old way" but making them horizontally scalable.

Amazon prefers to acknowledge that network partitions exist and try to get you to do things "the new way" that deals with that failure case in the software instead of trying to hide it.

I'm not saying either system is better than the other, but doing it Google's way is certainly easier for Enterprises that want to make the move, and why Amazon is starting to break with tradition and release products that let you do things "the old way" while hiding the details in an abstraction.

I've always said that Google is technically better than AWS, but no one will ever know because they don't have a strong sales team to go and show people.

This release only solidifies that point.

Re:Introducing Cloud Spanner, a Global Database Service by dietrichepp on Tuesday February 14, 2017 @02:06PM (#13646146)
This isn't entirely accurate. BigTable was Google's earlier cloud database, and it's certainly non-traditional, and you have to build your application without traditional consistency guarantees, the way you describe.

Spanner doesn't exactly hide the details, but it lets you make transactions that span multiple shards. You still eat the cost of the transaction, you're just free from having to implement it at the application level, which is a more difficult and error-prone way of doing things. The bottom line is that if you need consistency, it needs to be implemented somewhere in your stack. If you don't need consistency (analytics workloads come to mind) then you have more flexibility with your database.

Disclosure: Google employee, reconstructing what I know from published information.

Re:Introducing Cloud Spanner, a Global Database Service by q3k on Tuesday February 14, 2017 @09:33PM (#13649254)
Same for GAE, or now GKE (basically microservices before microservices were a thing) vs. EC2. GAE was pushing a fairly non-traditional architectural model while others were just trying to provide cloud-based VMs.

Discolsure: Also a Google employee, also reconstructing.

Re:Introducing Cloud Spanner, a Global Database Service by user5994461 on Tuesday February 14, 2017 @03:03PM (#13646618)
Amazon: Create usual services and sell them.

Google: Make unique products that push the boundaries of what was previously thought possible.

Amazon: Don't care about inefficiencies and usage. Inefficiencies can be handled by charging more to the clients, usage doesn't matter because the users are mostly the clients and they don't feel their pain.

Google: Had to make all their core technologies efficient, performant, scalable and maintainable or they couldn't sustain their business.

Re:Introducing Cloud Spanner, a Global Database Service by edblarney on Tuesday February 14, 2017 @04:33PM (#13647411)
Not fair.

Amazon: IaaS

Google: PaaS

Amazon is philosophy is being 'close to the metal' to allow Enterprise customer to migrate 'regular apps' into a 'regular environment' in the cloud.

Most of Google's offerings are (at least were) novel, but proprietary ways of doing specific things.

Amazon is not a laggard: they have provided a number of interesting and useful 'helper' things to facilitate IaaS - as well as a number of 'pure cloud' type things.

Amazon is very, very customer focused. Their products come from customer demands.

Google often 'cool things they've done internally' and exposes them, hoping that they might have some use-case in the rest of the world.

Google and Amazon are equally interested in profit.

Re:Introducing Cloud Spanner, a Global Database Service by computerex on Tuesday February 14, 2017 @11:21PM (#13649661)
Google is IaaS, their PaaS offering (App Engine) never gained much traction AFAIK. I also find the comparison fair, Google is a software engineering company, Amazon is a sales/marketing company.
Re:Introducing Cloud Spanner, a Global Database Service by Xorlev on Wednesday February 15, 2017 @12:25AM (#13649935)
At what point do infrastructure services become the platform? IMO, between GKE, Spanner, BigQuery, etc., it's basically a PaaS for non-trivial applications.
Re:Introducing Cloud Spanner, a Global Database Service by computerex on Wednesday February 15, 2017 @01:00AM (#13650069)
Re:Introducing Cloud Spanner, a Global Database Service by aanm1988 on Tuesday February 14, 2017 @03:46PM (#13646989)
> Google: Had to make all their core technologies efficient, performant, scalable and maintainable or they couldn't sustain their business.

Which Amazon totally didn't have to do with their firehose of cash?

Re:Introducing Cloud Spanner, a Global Database Service by user5994461 on Tuesday February 14, 2017 @04:15PM (#13647253)
Google has to support Google, Youtube and many of the most resources intensive services in existence on Earth. They needed to be "efficient enough" to operate that, meaning incredibly efficient.

Amazon runs nothing, it's an outsourcing firm. They needed to make services "good enough" to be sold. If a service is somewhat inefficient, it just charges the clients more to cover the costs.

Technologies reflect the business they were created in.

Re:Introducing Cloud Spanner, a Global Database Service by aanm1988 on Tuesday February 14, 2017 @04:22PM (#13647318)
> Amazon runs nothing, it's an outsourcing firm.

What the fuck are you talking about. It's one thing to say AWS services are "good enough" but "Amazon runs nothing" is a ridiculous statement.

Re:Introducing Cloud Spanner, a Global Database Service by boulos on Wednesday February 15, 2017 @01:53AM (#13650236)
Agreed, and I work on Google Cloud. We may have different styles and core businesses, but I wouldn't say "eBay ran nothing" either. Logistics alone is a super fascinating space!
Re:Introducing Cloud Spanner, a Global Database Service by ksec on Monday February 20, 2017 @05:32AM (#13686148)
Interesting. But Do Amazon actually do much Logistics? I thought they are much more Warehouse and outsourced delivery to DHL and UPS.
Re:Introducing Cloud Spanner, a Global Database Service by jplahn on Tuesday February 14, 2017 @05:38PM (#13647944)
You do realize the way most services get into AWS is that they're first built in the retail side of Amazon (without any thought towards AWS) and then once people realize it's effectively solving an actual problem, it's rebuilt for AWS. Having to support Amazon retail is a pretty demanding stress test -- I'm not sure why you're getting this notion that Amazon doesn't run anything. I should think handling Black Friday alone would count for something..
Re:Introducing Cloud Spanner, a Global Database Service by parasubvert on Tuesday February 14, 2017 @09:21PM (#13649193)
That is something of a myth. AWS was created and evolves completely separately from retail, which didn't really use it in anger until 2010ish. Retail is effectively a large customer to AWS. They're very good at watching what customers are doing in general.
Re:Introducing Cloud Spanner, a Global Database Service by discodave on Monday February 20, 2017 @11:47AM (#13688695)
The 'rebuilt for AWS' phrase is key.
Re:Introducing Cloud Spanner, a Global Database Service by computerex on Tuesday February 14, 2017 @11:26PM (#13649688)
No one is saying amazon doesn't test their stuff. The argument here is that Google is inherently a more technical company, which is a fair comparison. Their products are more technical. Ad Sense, Gmail, YouTube are incredibly technical products due to their scale, and the argument here is that nothing of similar technicality exists in Amazon's core business, which I think is totally fair.
Re:Introducing Cloud Spanner, a Global Database Service by mavelikara on Wednesday February 15, 2017 @03:04AM (#13650482)
> The argument here is that Google is inherently a more technical company, which is a fair comparison.

I suspect that Google knows this, and their reputation for have poor customer support and sales comes from that knowledge.

Re:Introducing Cloud Spanner, a Global Database Service by jedberg on Tuesday February 14, 2017 @03:07PM (#13646654)
Yeah that's all pretty accurate too. :)
Re:Introducing Cloud Spanner, a Global Database Service by dcosson on Tuesday February 14, 2017 @04:55PM (#13647564)
I would phrase it as:

AWS prioritizes building blocks that support very high throughput and avoid leaky abstractions at all costs, and they're happy to push forward as long as these criteria are met. IMO they really succeed at this goal. Minus specific bugs that they're generally good about acknowledging, their services reliably do what they say they're going to. And they definitely solve a lot of problems for you, even if sometimes you're still required to get further into the weeds than you might want.

I'll buy that Google Cloud is better at questioning underlying assumptions and sometimes succeeds in releasing higher-level abstractions than AWS without any leakiness (a great example of this now being Spanner vs Aurora). It also feels to me that with releases like this Google is leveraging the full value of their own experiences running their services, and seems to be more advanced than amazon in some areas so this has a lot of value, whereas AWS seems to build a broader range of products with a specific customer in mind which is not necessarily themselves (e.g. all of their move your on-prem stuff to the cloud helpers).

If you consider Spanner vs Dynamo, it definitely matches up as Google wrapping the old way and Amazon forcing a new way (though to be fair, Dynamo was released 5 years earlier). But on the other hand considering Spanner vs Aurora, Amazon is the one embracing the old way with full MySQL & Postgres compatibility whereas Spanner sounds like a pretty dramatically different subset of SQL in not supporting insert and update statements. It's a very reasonable compromise for basically getting to ignore the CAP theorem, but it is a a significant difference that every developer will have to learn.

Re:Introducing Cloud Spanner, a Global Database Service by vgt on Tuesday February 14, 2017 @06:15PM (#13648221)
I would probably compare Dynamo to Google Datastore (released.. 8 years earlier?), or even to Bigtable, which went GA last summer. I'm not sure Spanner matches up with anything AWS at this point.

(work on google Cloud)

Re:Introducing Cloud Spanner, a Global Database Service by ma2rten on Tuesday February 14, 2017 @10:31PM (#13649455)
I think aurora is the closest.
Re:Introducing Cloud Spanner, a Global Database Service by u320 on Tuesday February 14, 2017 @01:52PM (#13646028)
The "old way" was sacrificing functionality such as transactions and joins to get scalability (BigTable, DynamoDB).

Google tried that a decade ago and found it lacking, this is why Spanner exists in the first place.

Re:Introducing Cloud Spanner, a Global Database Service by jedberg on Tuesday February 14, 2017 @01:55PM (#13646063)
Well, I'd say the "old way" is SQL with joins and schemas and transactions, and the "new way" is KV with eventual consistency.
Re:Introducing Cloud Spanner, a Global Database Service by rrdharan on Tuesday February 14, 2017 @02:30PM (#13646373)
Chronologically, we have: SQL -> NoSQL -> NewSQL

You're both right.

Re:Introducing Cloud Spanner, a Global Database Service by takeda on Tuesday February 14, 2017 @01:51PM (#13646014)
I'm not sure if what you said applies, they have severe restrictions and spanner offers subset of MySQL functionality which is already bare compared to other databases. Changes can be done by primary key only, so it almost feels like a KV store that can do joins...

I don't think it's easy to port existing applications to use it and in the end you will still need to accommodate shortcomings in your application.

Re:Introducing Cloud Spanner, a Global Database Service by jedberg on Tuesday February 14, 2017 @01:56PM (#13646068)
That's a fair assessment. But I'm assuming they will make it do more "SQL" things in the future. I could be wrong though.

Either way, they are trying to abstract away having to think about eventual consistency with this offering.

Re:Introducing Cloud Spanner, a Global Database Service by cavisne on Tuesday February 14, 2017 @11:13PM (#13649631)
Spanner doesn't use SQL for writes it seems, so its still a significant rewrite for legacy applications (especially ones that dont use an ORM).

The thing thats really different here is Google are basically saying, heres this awesome system, yes it has obvious risks from partitioning, we are going to stake our reputation on those partitions not happening.

In contrast AWS are saying, this is DynamoDb, its really limiting but because of those limitations it should be pretty reliable as long as you write your application correctly.

It will be interesting to see if Microsoft and Amazon have to follow Google's lead here.

Re:Introducing Cloud Spanner, a Global Database Service by nickpsecurity on Wednesday February 15, 2017 @12:24PM (#13653628)
"I'm not saying either system is better than the other, but doing it Google's way is certainly easier for Enterprises that want to make the move, and why Amazon is starting to break with tradition and release products that let you do things "the old way" while hiding the details in an abstraction."

No, they said this in the F1 RDBMS or Spanner papers. They originally did the NoSQL, eventual-consistency type of stuff. This had app developers required to do a lot of work to avoid problems that model can create. Apparently, even their bright people had enough problems with it that they decided to eliminate or simplify that situation with stronger consistency. Took some brilliant engineering but now they have a database easy to use as old model with advantages of newer ones.

If anything, they learned some hard lessons with a good solution to them. Now, they're offering it to others. I was hoping they'd do this instead of keep it internal only. F1 and Spanner are amazing tech that could benefit many companies.

Re:Introducing Cloud Spanner, a Global Database Service by inopinatus on Tuesday February 14, 2017 @01:46PM (#13645982)
People used to make similar comparisons between the Russian and American space programs.
Re:Introducing Cloud Spanner, a Global Database Service by jedberg on Tuesday February 14, 2017 @01:53PM (#13646044)
Oh yeah? Which was which in that comparison? I'm not familiar with that.
Re:Introducing Cloud Spanner, a Global Database Service by WorldMaker on Tuesday February 14, 2017 @02:22PM (#13646303)
Presumably a reference to the classic "Pencil myth": Americans when faced with a need to write things down spent millions of dollars inventing a low gravity ballpoint pen and Russians just used a pencil.

As cutesy of a sentiment as it is, it's also full of misconceptions. The pens were invented by an American corporation that wanted better pens to sell in general (a smoother flow in a pen, regardless of gravity/orientation, is a better pen), and they saw a good opportunity to market the pen to NASA for use in space. Both NASA and the Russians used pencils in space, but the problems with pencils is the flakes can pollute an environment pretty quickly in low gravity and the pens turned out to be a much better solution. (So far as I've heard, every space agency these days buys similar pens.)

Re:Introducing Cloud Spanner, a Global Database Service by inopinatus on Tuesday February 14, 2017 @03:17PM (#13646725)
You presume wrong. Pens are not a space program.

I meant the differences in design philosophy that permeate aerospace engineering on both sides. Russian, built ugly but for strength and longevity. American, built for high capability with finesse and finer tolerances. The emergent properties of these different principles explain why Soyuz is still a preferred launch vehicle, but it was the Americans who got to the moon and operated the STS.

Amazon is more like the Russians: built in the knowledge that things fail, but less magical as a result. Google is more like the Americans: remarkable technology, you just need a herd of geniuses to run it.

Re:Introducing Cloud Spanner, a Global Database Service by WorldMaker on Tuesday February 14, 2017 @04:10PM (#13647223)
Sorry if you think I made the wrong presumption, but I've seen the "pencil myth" trotted out many times as a "definitive" example of precisely the dichotomy that you have spelled out. (A low gravity ballpoint pen is high capability that requires finesse and fine tolerances; a pencil is ugly but strong and typically exhibits longevity... and so forth.)

It is an interesting analogy this dichotomy you see in the design philosophies (both between the space programs and the mega-corporations), but perhaps my point, if I were attempting a point, is to beware of false dichotomies.

Re:Introducing Cloud Spanner, a Global Database Service by inopinatus on Tuesday February 14, 2017 @08:57PM (#13649102)
You can see the same in comparing the military aircraft design. Both schools are ingenious, but it is applied in different ways. There are some excellent comparisons connecting principles to outcomes on Quora.
Re:Introducing Cloud Spanner, a Global Database Service by hueving on Tuesday February 14, 2017 @01:44PM (#13645972)
>and release products that let you do things "the old way" while hiding the details in an abstraction.

However, by 'abstracting' this away, you're not being forced to think about failure domains. If there is ever a massive country-wide connectivity break to the wider Internet (feasible for lots of people inside censored countries), you'll be pretty pissed when you can't use the DB services for your servers in the Google-local datacenter that you still have connectivity to because it can't get quorum.

Re:Introducing Cloud Spanner, a Global Database Service by dietrichepp on Tuesday February 14, 2017 @02:10PM (#13646186)
Cloud Spanner is currently a regional service, not a global service. So you would only lose availability for failures within the region.
Re:Introducing Cloud Spanner, a Global Database Service by jedberg on Tuesday February 14, 2017 @01:54PM (#13646050)
Exactly my point. I would say I personally prefer the Amazon way of forcing you to think about these things.
Re:Introducing Cloud Spanner, a Global Database Service by boulos on Tuesday February 14, 2017 @04:20PM (#13647298)
I'd encourage you to read the F1 paper and the Spanner paper if you haven't already. The big thing that stood out to me is that Google started that way (with BigTable) where you'd roll your own availability and transactions. It didn't go well.

That's the motivation behind both Spanner and F1: take the experience of how painful it is to do transactions on a Regional or Global level, and never make individual teams do it again.

I see it a bit like "Don't roll your own crypto". Clearly some people are exempted from it, but you better be able to tell me why you get an exception.

Disclosure: I work on Google Cloud and want you to pay for Spanner :).