First of all, we have a few modifications in Django itself that make Django more flexible. What exactly are you thinking of that we're missing?

Regarding SQLCompiler: Maybe the name is just misleading? We had several attempts at this problem and one of them was a complete refactoring of the ORM as you try to suggest here, but in the end we had to face the facts: We'd reproduce pretty much the same functionality that's already there. It would just be named Query instead of SQLQuery and it would take a huge amount of effort to get there. So, we decided instead to reuse as much of the code that just happens to be named "SQLSomething" as possible. And IMHO this is the best of the approaches we've taken. It's simple and it works really well, as you will soon see for yourself. Then we'll talk again.

BTW, we also tried to work with the Django team, but in the end it was all discussion with the hidden message "show us that it works". That's what we are doing now.

When you are naming something like the SQLCompiler "SQLCompiler" even if it fits non-relational database's needs than naming it "SQLCompiler" was wrong!

As I said on reddit, the name isn't misleading. SQLCompiler is used for generating SQL, and assumes relational database concepts like joins, table aliases, and aggregation.

Now, here comes the surprise: That's exactly what we want to emulate with Django nonrel. It's exactly the feature set we badly want to have. This is the reason why the SQLQuery/SQLCompiler stuff fits so well (though, you're right, we're only using a fraction of SQLCompiler's code, so we could actually implement a simplified base class which is used by both, but we can discuss this later on the django-developers group).

For the record, here's an alternate plan for non-relational database support in the Django ORM (I'm considering applying for GSOC with it, in place of the template compilation proposal I already put forwards): http://paste.pocoo.org/show/198394/

If you want to work on this bigger ORM refactoring I'm available for mentoring help with the App Engine backend (most probably others, too) and with non-relational DB concepts in general (if you need/want my help there). Like I said, we initially wanted to do something along the lines you suggest, but we couldn't really justify the significantly increased amount of work compared to the current approach. If you think it's worth the effort then please go for it.

In any case, the final nonrel backend API won't be much (or any) different from what we have now, so if someone wants to write a backend, please do so.