This is my experience as well. At the end of the day, most line of business applications can just use SQLite and be done with it. One project, one executable, one database, one model, one machine. Very rarely does the business actually need the kinds of fantastical things that are routinely bandied about on hackernews. Now, that isnt to say these things arent fuckin awesome and that we should stop talking about them, we just need to be realistic about the scope and applicability of our tools.
In terms of a universal API, we are kind of doing this in our shop. We are building 1 unified domain model, which is approximately in 3rd normal form. All of our software is required to use this model to represent the facts/state of whatever is happening. Changes to this model go through a semi-formal review process, just like you would expect with an old school SQL setup in a big shop. Our model can be mapped one-way to SQL for purposes of online business rule evaluation or offline reporting.
The benefits of having 1 standardized model are really cool after its been in place for a while. You will notice your non-technical project managers developing an awareness for which top-level types may be implicated in certain matters. Having one slow-changing way to represent the business at hand makes it really easy for people to start picking things up. When its scattered to 50 repositories and 10 different languages, this is a bit trickier.
If we want a robust model, we have to include uncertainty within it. As long as computers fail by being sure about the error, a human is always needed to fix the problem.
It would have to be an almost philosophically mathematical standard. How we do that when half the world measures physics by completely different metrics than the other one?
At least we have Unicode. God bless Unicode. Solving the problem from a computer-linguistic viewpoint: Let's get the language into the computer and we can worry about what it means later!
And they're even more limited than relational databases when it comes to the ability to scale out, or have good consistency semantics without deadlocks. Just implementing a basic mutex on a (general) filesystem is nigh-on impossible. And while AFS is impressive, there still aren't really any true master-master filesystems, much less filesystems that can autoscale.
I think it's far more likely we'll just discover that, indeed, no general solution exists, and that humans must take personal responsibility for tending to their digital gardens. We certainly have a lot of spam to weed out.
There are limits to what you can do safely with autoscaling, sure. But a datastore without that capability will never be a universal datastore that can cover all use cases.
The problem you're dedscribing is political: How to get everyone across cultural and language barriers agree upon what the bits mean?
There is no technical obstacle to everyone accepting a universal API. In many ways, we're on our way there already. The Internet is full of standards and protocols. Yet it shunts knowledge around the world caring only of a few numbers.
It might be something fundamental, like that each aspect of reality has to be attended to by a sentience. Like every person is responsible for maintaining some part of the universal standard. Maybe general-purpose-anything is flawed as a concept, and we must come to understand that what we seek cannot be automated in a meaningful way.
Maybe in the age of Turing et al there was so little knowledge it all seemed computable. Now we're drowning in it.