News for hackers. Stuff that matters.
Re:We Can Do Better Than SQL by tathougies on Thursday May 09, 2019 @03:54PM (#19871706)
> I use Postgres in the first place because there are so many improvements available on top of SQL.

Most of Postgres is standard SQL. It's just that most non-Postgres databases do not implement standard SQL very well.

Re:We Can Do Better Than SQL by brightball on Thursday May 09, 2019 @04:16PM (#19871953)
The various extensions / plugins that allow for custom data types, indexes, use of multiple programming languages to write functions, ability to use a foreign data wrapper to connect to Redis and build a VIEW out of the result or push data out to Redis/Memcached with a database function, varieties of powerful search capabilities, etc

It's got a lot of stuff going on in there.

Re:We Can Do Better Than SQL by cagmz on Friday May 10, 2019 @10:34PM (#19884163)
> ability to use a foreign data wrapper to connect to Redis and build a VIEW out of the result or push data out to Redis/Memcached with a database function,

This sounds amazing. Does anyone have a link to some docs for this?

Re:We Can Do Better Than SQL by ken on Thursday May 09, 2019 @04:43PM (#19872236)
That "most" is a pretty loaded word! Last I checked, even CREATE INDEX is not part of any ANSI or ISO SQL specification. (That's why every RDBMS has such different features and syntax for this.) Good luck building a system with just "standard SQL".
Re:We Can Do Better Than SQL by paulddraper on Friday May 10, 2019 @05:05AM (#19876227)
> even CREATE INDEX is not part of any ANSI or ISO SQL specification

That's correct, and that's why PostgreSQL has ADD CONSTRAINT.

The constraints describe the actual schema of the data; INDEX is an DBMS-specific implementation* for improving performance (including index types, etc.).

* Those since the standard does not yet cover some thing partial unique constraints, these have to be done as INDEX in PostgreSQL.

Re:We Can Do Better Than SQL by ErwinSmout on Saturday May 11, 2019 @03:57PM (#19888069)
SQL has nothing to say about the layer of physical implementation, and that's where INDEX belongs (as does STOGROUP [DB2] and what have you).

That's not a bug, that's a feature. Deriving from the deliberate intent to make physical independence (which was simply nonexistent at the time the model was conceived) a reality.

Re:We Can Do Better Than SQL by mbreese on Thursday May 09, 2019 @05:32PM (#19872652)
Why should it be? Indexing is an implementation detail for each RDBMS, so it doesn’t make sense to add it to the language spec.
Re:We Can Do Better Than SQL by garmaine on Thursday May 09, 2019 @11:29PM (#19874822)
It does make sense when the goal of the language spec is interoperability, and it's something that everyone has to do, in practice, no matter the platform.

Call it a "hint", if you will, the meaning of which is implementation defined. But standardize the syntax for pete's sake.

Re:We Can Do Better Than SQL by ErwinSmout on Monday May 13, 2019 @02:39AM (#19896817)
Interoperability is achieved if the same query with the same inputs yields the same result. "In the same amount of time" (which is the part that indexes are aimed at) is manifestly not part of that picture, and that's by design.

And "hints" in the language are not exactly going to improve "interoperability" if their meaning is still allowed to be implementation-defined, are they ?

Re:We Can Do Better Than SQL by ken on Friday May 10, 2019 @12:38AM (#19875089)
That sounds backwards to me. It's only lack of standardization that forces it to be an "implementation detail". To any application that needs to be able to query data in reasonable time, it's fundamental and necessary functionality.
Re:We Can Do Better Than SQL by inferiorhuman on Thursday May 09, 2019 @03:59PM (#19871774)
Most of Postgres is standard SQL. It's just that most non-Postgres databases do not implement standard SQL very well.

Sure, but the non-standard enhancements like JSON support are part of what sets Postgres apart from the competition IMO.

Re:We Can Do Better Than SQL by wtetzner on Thursday May 09, 2019 @04:31PM (#19872121)
SQL:2016 includes support for JSON: https://en.wikipedia.org/wiki/SQL:2016
Re:We Can Do Better Than SQL by ioltas on Thursday May 09, 2019 @08:59PM (#19874105)
Postgres 12, whose development has reached feature freeze and is in a stability period, has added some support for JSON path as of SQL:2016: https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit...

The release is planned for roughly next September/October.

Re:We Can Do Better Than SQL by inferiorhuman on Thursday May 09, 2019 @04:46PM (#19872267)
Interesting, I wonder which databases actually implement SQL:2016 (and JSON as standardized). JSON support in Postgres predates SQL:2016 in any case.
Re:We Can Do Better Than SQL by crazysmoove on Thursday May 09, 2019 @05:51PM (#19872786)
I think Oracle does. My impression is that PostgreSQL sort-of "invented" JSON support in the db, then Oracle (and probably others) added JSON support with a different syntax, and then Oracle got their version defined into the SQL standard. I'm half-guessing here, but it's based on what I've gleaned from Markus Winand's [0] excellent compatibility tables in his slides [1].

[0] https://modern-sql.com/

[1] https://modern-sql.com/slides/ModernSQL-2019-05-08.pdf, for example

(Edited for formatting.)

Re:We Can Do Better Than SQL by irrational on Thursday May 09, 2019 @04:33PM (#19872142)
Wait, what major relational databases don't support JSON? Oracle and Sql Server definitely do. Does MySQL not support JSON?
Re:We Can Do Better Than SQL by ken on Thursday May 09, 2019 @04:35PM (#19872157)
SQLite (the most popular RDBMS today) doesn't, unless you count extensions. And of course, none of them use the same syntax or functions or data types.
Re:We Can Do Better Than SQL by matharmin on Thursday May 09, 2019 @04:49PM (#19872284)
While it is technically an optional extension in sqlite, it is part of the main source distribution, and very easy to enable.
Re:We Can Do Better Than SQL by nbevans on Friday May 10, 2019 @03:27AM (#19875809)
And why wouldn't you count SQLite extensions? Its JSON1 extension is practically ubiquitous.
Re:We Can Do Better Than SQL by marcolussetti on Thursday May 09, 2019 @05:41PM (#19872711)
If I recall correctly, Oracle's support for JSON is not at the same level as PostgreSQL. By that I mean seamlessly store JSON as a data type like any other. In Oracle JSON is stored as VARCHAR/CLOB and then you implement tests to validate whereas in PostgreSQL and MSSQL is its own data type.
Re:We Can Do Better Than SQL by GordonS on Friday May 10, 2019 @02:38AM (#19875566)
Similar story with SQL Server. I believe JSON support is a bit better in SQL Server 2019, but still far lacking behind that of Postgres.
Re:We Can Do Better Than SQL by lefred on Thursday May 09, 2019 @06:20PM (#19873000)
Hi, MySQL supports JSON (it has a native JSON datatype) but it also support NoSQL (CRUD operations). For more details, check https://www.slideshare.net/lefred.descamps/pyconx-python-mys... or https://www.slideshare.net/lefred.descamps/oracle-code-roma-...
Re:We Can Do Better Than SQL by jade12 on Thursday May 09, 2019 @04:12PM (#19871919)
Doesn't every major RDBMS support JSON out of the box these days?
Re:We Can Do Better Than SQL by zaarn on Thursday May 09, 2019 @04:34PM (#19872151)
(Last I checked atleast) JSON isn't as well supported on MySQl or MSSQL compared to PG.

In PG, a JSON column is so well integrated that you can do all sorts of crazy stuff (indices over JSON queries is my favorite). You could build an entire RDBMS on top of PG's JSON column.

Re:We Can Do Better Than SQL by atwebb on Thursday May 09, 2019 @04:52PM (#19872313)
MSSQL indexes and provides dot notation/object query from 2016 forward, schemas are supported as well.
Re:We Can Do Better Than SQL by jtms on Thursday May 09, 2019 @08:00PM (#19873749)
I feel like Postgres is so powerful I could damn near build an entire web app backend with JUST Postgres (i'm only sorta kidding here). If that's standard SQL, well then I really like standard SQL :-)
Re:We Can Do Better Than SQL by CuriouslyC on Thursday May 09, 2019 @09:19PM (#19874220)
I've done pretty much this a few times, and it's amazing if you're working in the context of enterprise data systems that need to provide extensive capabilities to a "small" userbase (i.e. concurrent in the thousands). You just need a small shim layer for security, to transform results sets, and handle browser -> database connectivity.

Postgrest works very well as this shim layer, though I've moved on to writing sql directly in the client and communicating through a web socket shim. In terms of reducing code complexity and improving performance this is absolutely unbeatable, you just need to parse incoming sql to sanitize it and make sure there is no role escalation. Because of postgres's foreign data wrappers this method can provide a consistent surface for basically all your enterprise data. The only gotcha with FDWs is that some of them don't "push down" many query clauses, so you end up doing much slower queries on the remote system and filtering locally, which is terrible for obvious reasons. That being said, the FDWs are pretty much all open source, so you can just implement push down support for those missing clauses yourself.

Re:We Can Do Better Than SQL by menssen on Thursday May 09, 2019 @11:43PM (#19874869)
> you just need to parse incoming sql to sanitize it and make sure there is no role escalation

"Just"? I guess I'm skeptical of a statement that begins "you just have to parse sql".

Is this actually easier than I'm imagining it? I'd be curious to hear more about the security and authorization model of this approach.

Re:We Can Do Better Than SQL by CuriouslyC on Thursday May 09, 2019 @11:56PM (#19874918)
You don't have to do full statement parsing, you basically just have to do a limited parse that looks for the various ways that someone could execute a set role statement. As long as you don't let a user execute set role, you use db roles for user accounts, you have a reasonable statement timeout in place to prevent DOS, and your postgres security model is tight (the big gotcha is not to allow access to untrusted code), this approach works fine.

A good tool for this purpose is https://github.com/JavaScriptor/js-sql-parser as it will fail to parse complex statements that are likely to include an attack vector.

In terms of authorization, you can either create per user connection pools if using web sockets and log the user in directly that way (which makes things easy) or if you must use rest, use a single connection pool with a master user then use some form of token to tell the shim who to set role to before executing the query.

Re:We Can Do Better Than SQL by jtms on Thursday May 09, 2019 @10:51PM (#19874641)
Thank you for posting this. I have been thinking about using something along these lines for awhile now. Next project that fits the bill I will see about building a proof of concept implementation and go from there. Have you had any exposure to any of the other projects similar to PostgREST that you have any thoughts about?
Re:We Can Do Better Than SQL by CuriouslyC on Thursday May 09, 2019 @11:46PM (#19874882)
I've played with Graphile, it's not a bad choice if you are already invested in GraphQL. I'm not a GraphQL fan for a number of reasons, but it's a definitely a solid project.
Re:We Can Do Better Than SQL by ar-jan on Friday May 10, 2019 @04:56AM (#19876189)
I'm keeping an eye on https://github.com/aquametalabs/aquameta "Web development platform built entirely in PostgreSQL"
Re:We Can Do Better Than SQL by jtms on Thursday May 09, 2019 @10:56PM (#19874662)
Graphile looks quite intriguing! any experience with it? Recommended?
Re:We Can Do Better Than SQL by dafrie on Friday May 10, 2019 @01:46AM (#19875378)
I would also have a look at Hasura GraphQLEngine. Have used it now for various projects and is extremely nice, especially with the superb subscriptions support and eventing for the odd requirement that cant be handled within Postgres
Re:We Can Do Better Than SQL by Munksgaard on Friday May 10, 2019 @01:19AM (#19875261)
I'm using Graphile in deployment at the moment, and it works quite nicely. Granted, we have a limited amount of users thus far, so I don't really know how well it handles big loads, but it allowed us to get off the ground with a PoC _really_ quickly.