News for hackers. Stuff that matters.
Re:Amazon Time Sync Service by drewda on Wednesday November 03, 2021 @06:29PM (#29101273)
Yes, which is why it's amusing in hindsight that for a decade everyone* outside Google was forcing all* their distributed data tasks into the MapReduce paradigm, without considering alternative approaches like the one used by Spanner.

* slight exaggerations, I know

Re:Amazon Time Sync Service by mrep on Wednesday November 03, 2021 @06:38PM (#29101373)
I'm not sure how you think a distributed data processing technology would "fake-out" other companies when building/choosing database technology. They are totally different problem sets.

MapReduce does not have a set in stone data source/sink and can use multiple things like bigtable and spanner so they are complementary technologies.

Re:Amazon Time Sync Service by rp1 on Wednesday November 03, 2021 @10:07PM (#29102915)
I think the parent commenter might be referring to systems like Hive or HBase built on top of Hadoop and do have a lot of overlap with a large scale database system.
Re:Amazon Time Sync Service by drewda on Thursday November 04, 2021 @01:30AM (#29103876)
Exactly, thanks. "MapReduce paradigm" wasn't precise, and I can see why that makes everyone want to give a distributed systems 101 lecture in this comment thread :)
Re:Amazon Time Sync Service by quin3 on Thursday November 04, 2021 @01:51AM (#29103948)
HBase isn’t really related to MapReduce though, more akin to BigTable.
Re:Amazon Time Sync Service by quin3 on Wednesday November 03, 2021 @06:38PM (#29101376)
It’s not even related. No one was running OLTP workloads as MapReduce jobs at any point.
Re:Amazon Time Sync Service by LaserToy on Wednesday November 03, 2021 @07:15PM (#29101756)
Spanner didn’t exist in 2012.
Re:Amazon Time Sync Service by jeffbee on Wednesday November 03, 2021 @07:56PM (#29102112)
Maybe, but this 2010 presentation mentions it.

https://cloud.google.com/files/storage_architecture_and_chal...

Re:Amazon Time Sync Service by dekhn on Wednesday November 03, 2021 @08:27PM (#29102323)
at that time, only aristocrats could use spanner.
Re:Amazon Time Sync Service by teraflop on Wednesday November 03, 2021 @08:20PM (#29102277)
Yes it did. Google published a paper about it in 2012, and claimed that at that point it had been in development for 5 years and in production for more than 1.
Re:Amazon Time Sync Service by LaserToy on Wednesday November 03, 2021 @09:42PM (#29102773)
Mmm, I worked at google in 2012 and F1 was in active development.

I’ve never heard of Spanner internally. Maybe it was in development, but it was not in use.

Edit: went and read more. Looks like Spanner existed but didn’t have sql, so it wasn’t what it is today. And looks like I don’t remember things any more.

Re:Amazon Time Sync Service by ithkuil on Thursday November 04, 2021 @05:36AM (#29105053)
at least by 2013 it had some basic SQL (although much more limited subset that one usable today); if you needed more you would be using f1; IIRC I used spanner (without F1) in production around 2013