News for hackers. Stuff that matters.
Re:Cloud Spanner is now production-ready by erikpukinskis on Tuesday May 16, 2017 @04:31PM (#14353047)
General computer science-y kind of software philosophy question:

Should we really be thinking of sharding as something you can outsource to the infrastructure layer?

It just seems like having a stance about how the data in your specific domain naturally shards is probably going to pay off not even the long run, but immediately in terms of sanity checking your information architecture.

And I wonder if in 2017 we haven't gotten to the point where a cloud application should just shard, because we're trying to think about software as something that runs on a transient instance with access to a subset of data, appearing and reappearing, not a giant box with everything on it.

I get that Google engineers are darn close to abstracting away that "giant box with everything on it" behind an API, but I guess I'm asking if that's really how we should be thinking about our code.

Re:Cloud Spanner is now production-ready by scott00 on Tuesday May 16, 2017 @05:40PM (#14353547)
As with basically all software design questions, the answer is "it depends".

Writing a LOB app with 4 expected users all located in the same location, and expected storage requirements of a gigabyte a year? Go with a traditional DB, add replication if you need high availability, make and test your backups, done.

Writing the new Facebook weknoweverything app that captures smartphone audio and video continuously for all users at all times, automatically transcribes all conversations and produces AI-driven summaries of all video, all of which are saved forever and highly searchable? You're going to need a highly customized data storage architecture to have a prayer of keeping up with that data.

Somewhere in between? Some of those are good candidates for Spanner. Some aren't.

Re:Cloud Spanner is now production-ready by degenerate on Tuesday May 16, 2017 @05:24PM (#14353419)
I agree with your line of thinking. But at really big companies with hundreds of small teams, all working on separate projects, when the boss says to hook all that stuff together I'm sure a solution like Spanner looks really delicious and cost-effective.

For you and me working with data we already understand and added to our project piece by piece, we can safely shard and compartmentalize.