News for hackers. Stuff that matters.
Re:SQL Databases Are An Overapplied Solution (And What To Us... by peterwwillis on Wednesday February 17, 2010 @06:07PM (#1132906)
if by "solve" you mean "completely redesign modern RDBMS's" you may be waiting for a while. i'm no expert, but my limited understanding is that most of these services provide a singular interface to a database. to be "automatic infinite horizontal scaling" they'd need to support an infinite number of interfaces on any of these commodity servers. i could be wrong, because again i don't fully understand them, but i think that means basically implementing the equivalent of an 'intelligent' distributed parallel fault-tolerant file system. at that point i figure you could add "lots of glue" (heh, understatement?) to MemcacheDB, MemcacheD, MemcacheQ and some other crap and get something similar. but i could be crazy.
Re:SQL Databases Are An Overapplied Solution (And What To Us... by peterwwillis on Thursday March 11, 2010 @01:17PM (#1184441)
http://ostatic.com/blog/a-bright-future-for-drizzle

I guess this is old news but this is basically a much moreand flexible form of what I was thinking of (I was only considering the infrastructure/fault-tolerant portion of an RDBMS). You could definitely design a well-scaling horizontal cluster using this design; even if the stock components don't support what you need you just modify stuff outside the microkernel.

Re:SQL Databases Are An Overapplied Solution (And What To Us... by marshallp on Wednesday February 17, 2010 @06:43PM (#1132971)
The google appengine datastore is moving in the direction of being a full sql database, and I bet oracle have people working on it (because appengine, salesforce and aws is a threat to their business), so I don't think it's all that far-fetched.
Re:SQL Databases Are An Overapplied Solution (And What To Us... by csytan on Wednesday February 17, 2010 @09:23PM (#1133238)
Appengine's GQL is a very limited subset of SQL. The only queries which can be performed are those that can scale well, so it's doubtful that even simple things like JOINs will be supported in the future.

http://code.google.com/appengine/docs/python/datastore/gqlre...

Re:SQL Databases Are An Overapplied Solution (And What To Us... by marshallp on Wednesday February 17, 2010 @09:41PM (#1133265)
Appengine datastore works by creating indexes for every query. In relational databases you make joins execute fast by creating indexes. I don't see why joins couldn't be added to appengine.
Re:SQL Databases Are An Overapplied Solution (And What To Us... by davidnelson on Wednesday February 17, 2010 @10:06PM (#1133298)
I believe there's a few reasons. one is that the google file system blocks that bigtable runs on and "reads" for your query could be on any number of machines. been a while since I read the google architecture papers but I would recommend it to all. it is fascinating how they implemented gfs, bigtable, chubby lock, etc. the work around is essentially to write your own code that does what a database does - something to the effect of getting a list of keys for the records you are interested in and using that to then scan just the area in your data structure you want.
Re:SQL Databases Are An Overapplied Solution (And What To Us... by marshallp on Wednesday February 17, 2010 @10:30PM (#1133327)
It hasn't got anything to do with that. Creating an index on a join means that possibly hundreds of indexes need to be updated every time a you make a write - but that is where appengine would actually shine since those writes can be parallelized to hundreds of machines. It could easily make postgres/mysql/oracle redundant.