News for hackers. Stuff that matters.
Re:SQL as an API language by scrollaway on Thursday October 04, 2018 @04:48PM (#18143531)
GraphQL has a lot of advantages over SQL, API-wise; namely it's a lot more natural to describe what you want to get back. But without getting into that, the point is that you usually don't want to give direct data access, with all that entails.

You still want to be declaring the fields and properties of the data that users will be querying, and that usually sits a layer above your raw data.

You can make it a SQL view if you like, and give access to those instead. But it's just a lot of hassle to solve a solved problem in an unfamiliar way.

_______

Edit: Clarifying my stance on SQL views. I'm not dismissing it as a "bad tool", but there's good reasons it's not a popular one today. Writing SQL is not easy for everyone, and it's especially not easy for potential API consumers.

The best APIs are those you grok. If you grok SQL, congratulations, but most devs don't, and APIs are usually designed for most people.

Obviously, if SQL is the right tool for the job, use SQL. But don't start doing stuff like this for ideological reasons.

Re:SQL as an API language by tathougies on Thursday October 04, 2018 @05:13PM (#18143720)
I mean SQL views have been a thing for literally decades longer than GraphQL. If any technology should be accused of causing a 'lot of hassle to solve a solved problem', it's graphql, not SQL.
Re:SQL as an API language by atombender on Thursday October 04, 2018 @05:45PM (#18143931)
If your original data is flat and your app can consume it flat, no worries. But SQL is a worse fit for structured data, so you end up with an ORM on top, which is what GraphQL sort of is. Typically web apps are written with web technologies that work with structured objects, and you want something like {title: "Stapler", price: 1230, category: {id: "123", name: "Office supplies"}, mainPhoto: {url: "https://...", width: 600, height: 300}} back to feed directly in the UI.

Some databases (Postgres comes to mind) are better at letting you express this as a query which builds the structure using JSON functions in the SELECT part of the query. But it's not even close to ideal.

Re:SQL as an API language by just_myles on Thursday October 04, 2018 @06:03PM (#18144072)
"Some databases (Postgres comes to mind) are better at letting you express this as a query which builds the structure using JSON functions in the SELECT part of the query. But it's not even close to ideal."

Depending on which version of postgres you have(Want to say pre 9.5.) it can be a pain. There are various functions that allow a JSON output and another to make it human readable. With regards to performance I couldn't say but I imagine it would not be as fast as GraphQL.

Re:SQL as an API language by profalseidol on Sunday October 07, 2018 @06:00PM (#18162857)
In this context, what do you think of Datomic? Wish they make it the usual open source with a paid version with more features.
Re:SQL as an API language by atombender on Sunday October 07, 2018 @06:30PM (#18162973)
I don't know Datomic very well, but everything I've read makes it sound a bit like a modern take on an object database. Certainly interesting, but as long as it's commercial software I'm not going to touch it. Its close coupling (AFAIK) to Clojure is also not interesting to me.
Re:SQL as an API language by aaaaaaaaaab on Thursday October 04, 2018 @06:35PM (#18144286)
You don’t need an ORM, just a big JOIN.
Re:SQL as an API language by yen223 on Thursday October 04, 2018 @08:37PM (#18144807)
JOINs also return flat data structures, no?
Re:SQL as an API language by aaaaaaaaaab on Friday October 05, 2018 @04:23AM (#18146571)
Just iterate through the results and recover the hierarchical structure. It’s not exactly rocket science:

    rows = sql("SELECT * FROM parent LEFT JOIN child ON parent.id = child.parent_id")
    results = []
    for row in rows {
        if row["parent.id"] != results.last.id {
            results.add({ id: row["parent.id"], children: [], ... })
        }
        if row["child.id"] != null {
            results.last.children.add({ id: row["child.id"], ... })
        }
    }
Re:SQL as an API language by atombender on Friday October 05, 2018 @05:41PM (#18151964)
You have to handle arbitrarily nested hierarchies.

You also need to support referential equality — so that objects with the same type and ID map to the same in-memory JS instance — which requires building an identity map that can live across multiple paginated result set pages.

You have to support untangling the object trees for UPDATE statements, which will require tracking dirtiness to avoid unnecessary requests. Then something similar to deconstruct new objects into multiple INSERTs. Both require a knowledge of foreign-key relationships unless you run all constraints in DEFERRED mode.

Now you're building an ORM.

(Your query also makes assumptions about order and structure that you cannot make in the general case. The devil is in the details.)

Re:SQL as an API language by laszlokorte on Friday October 05, 2018 @08:10AM (#18147440)
but now it's no longer a declarative query. What if you want to include grand children? or if you want to fetch just one child but include it's parent and it's siblings of another type like a product, it's category and the categories tags:

   {id: 23, name: 'Trackpad', category: {id: 42, name:'Equipment', tags: [{id:...}, {id:...}]}}
Now instead of just adding two joins you have to build a custom loop with hand crafted conditions. Sure you can find a way to generalize that loop and put it into a library but still the query itself is separated from some kind of post processing that has be kept in sync.
Re:SQL as an API language by verbify on Friday October 05, 2018 @08:39AM (#18147617)
Nice bit of code.

Just to clarify, wouldn't you need to order by parent_id for the row["parent.id"] != results.last.id to work? Otherwise, say you're ordering by child.date_modified, you could end up with two identical parent_ids in your array?

Re:SQL as an API language by aaaaaaaaaab on Friday October 05, 2018 @10:52PM (#18153225)
Yeah, I forgot the ORDER BY clause, but you get the idea.
Re:SQL as an API language by dragonwriter on Thursday October 04, 2018 @05:36PM (#18143887)
> You can make it a SQL view if you like, and give access to those instead. But it's just a lot of hassle to solve a solved problem in an unfamiliar way.

It's kind of odd to use that to dismiss the solution that was established and widely available for 30 years longer, and which more people are probably familiar with.

Re:SQL as an API language by scrollaway on Thursday October 04, 2018 @06:03PM (#18144077)
See my edit. I'm not dismissing them, but the range of cases where they're the correct tool over a REST API, GraphQL, or something else... is very narrow.
Re:SQL as an API language by aidos on Thursday October 04, 2018 @05:31PM (#18143853)
I’m a huge proponent of sql, for me it’s virtually impossible to beat as a query language.

We played with wrapping our dB layer with graphql recently and I would say that the experience was pretty good. In part it’s because you’re working a layer up where you can have a more pluggable model. In the first run you can model your local dB structure, but it’s easy to glue more stuff into it from other sources and generally evolve the structure.

Re:SQL as an API language by davidgould on Thursday October 04, 2018 @08:51PM (#18144853)
HAVING is not the same thing as WHERE. WHERE filters the query to produce a result set. HAVING filters the result set.

    select id, count(*) - 1 as nduplicates
       from parts where color = 'red'
       group by id
       having count(*) > 1;
WHERE cannot filter on the count(*) because WHERE is applied during the query before the count is known.

I agree SQL is kind of warty, and it would be great if it were easier to compose from programs, but HAVING is not redundant to WHERE.

Re:SQL as an API language by irishsultan on Friday October 05, 2018 @07:31AM (#18147261)
But that's equivalent to the following query which doesn't use having, only select

    select * from (select id, count(*) - 1 as nduplicates
       from parts where color = 'red'
       group by id) _inner
       where nduplicates > 0;

Now obviously that doesn't read as easily, but WHERE can filter out the count(*) perfectly.
Re:SQL as an API language by aidos on Friday October 05, 2018 @04:04AM (#18146500)
I believe this should have been in response to its sibling. The set theorist in me agrees - the two operations are working on 2 different sets at 2 different levels of abstraction.
Re:SQL as an API language by davidgould on Sunday October 07, 2018 @01:00AM (#18158917)
Yes, replied to the off by one post.
Re:SQL as an API language by barrkel on Thursday October 04, 2018 @06:43PM (#18144337)
I find it very easy to imagine a better SQL.

Two things I'd fix:

* make it explicit when a join is expected to have 1:1 semantics vs 1:n, because it's very easy to end up with duplicated rows otherwise

* increase composability of the syntax; why do we need HAVING, why isn't WHERE enough? We have relational algebra, projections, filters, joins, folds, union, intersect etc. But the syntax of SQL is so idosyncratic, and at times antagonistic to the stream and set-like intuitions behind relational algebra.

Re:SQL as an API language by aidos on Friday October 05, 2018 @04:08AM (#18146513)
Regarding your first point, I’m trying to imagine what that would look like. So you mean something you could put on there to say (join x [1:1] on blah)? In general the mistake that’s easiest to make is to get duplicate rows from another join later, which this wouldn’t help with - but maybe I’m misunderstanding.
Re:SQL as an API language by barrkel on Friday October 05, 2018 @05:32AM (#18146807)
I don't really understand your comment about it always being join x+n that causes the problem instead of join x. What is different about later joins that is not true when the later join is the current join, when you get around to writing it? If joins for tables T1::T2 and T2::T3 are 1:1, they don't magically become 1:n when you do T1::T2::T3.

To do it right, you'd need to mark up the relations with expected cardinality (effectively unique constraints), so that the DB would be able to verify ahead of time whether a join is going to be 1:1 or 1:n. That would be a better solution than a join working fine up until the cardinality expectation is violated and it suddenly stops working.

If we had to stick with SQL syntax, it might be something like:

   select a.*, b.*
   from a
   join one b on b.id = a.b_id
(Unique constraint from primary key on b.id. It may still filter down the set of rows in a, but it won't duplicate.)

Or:

   select a.*, b.*
   from a
   join many b on b.some_key = a.some_key
(No unique constraint on b.some_key)

I think the interesting cardinality distinctions are 1:1, 1:n and 1:0.

(The above syntax is ambiguous with aliases, so it wouldn't fly as is. But it gives a flavour.)

Re:SQL as an API language by aidos on Friday October 05, 2018 @09:07AM (#18147763)
So the idea is that when you write the queries with these explicit join requirements, you'll want the query to fail if someone changes the schema later in such a way that the requirement is no longer met? If so, it seems like something that comes up so infrequently as to not warrant the noise, but maybe I'm still misunderstanding.

What I was saying was that often people have T1::T2 (1-1) and then throw in a later join T2::T3 (1-many) without considering the implications for the T1 (maybe they were doing a count(T1.id)). At least that seems to be an aspect that confuses developers newer to sql (that the many join is duplicating other parts of the data).

Re:SQL as an API language by barrkel on Saturday October 06, 2018 @03:40AM (#18154035)
It comes up quite a bit if you have repeated data in batches where the batches are a first class concept. Any relations need to be qualified by both normal fk id and batch id, or else you bring back across all batches.

Think something like measurements samples that come in an envelope with metadata, or account updates that come in a statement, that kind of thing.

Re:SQL as an API language by walshemj on Thursday October 04, 2018 @06:31PM (#18144265)
Why is writing SQL so hard for "most devs" I was moved to a Oracle based project (the management system for the uk's core internet) and only had a weeks training on PL/SQL.

I got a high performance award after that project was completed and I don't consider myself a Rockstar developer

Re:SQL as an API language by scrollaway on Thursday October 04, 2018 @06:40PM (#18144309)
You have a week's worth more experience of training than "most devs".

"Most devs" don't directly deal with SQL these days, because ORMs are good enough for day-to-day operations (and the non-day-to-day ends up being handled by people who actually know SQL). In fact, "most devs" rarely interact with databases. A lot of data sourcing is API driven these days.

Now, I don't claim to like or dislike that particular state. But you have two choices: You can innovate, which carries risks (of being wrong; of not selling your idea correctly; of reinventing the wheel; of making the same mistakes others did; ...). Or, you can play it safe, and create APIs that most people will know how to use without having to go through a 1-week course, because short as that may sound, that's 167.5 more hours than anyone will reasonably be willing to invest in understanding your API.

So at the end of the day: Use the right tool for the job. SQL for APIs is usually not the right tool.

Re:SQL as an API language by marcosdumay on Thursday October 04, 2018 @06:08PM (#18144109)
Out of curiosity (didn't had the time to get up to date on it yet), how does GraphQL let you describe your returning data better? Is it because it has hierarchical types (types and collections inside types) or it's something else?
Re:SQL as an API language by scrollaway on Thursday October 04, 2018 @06:41PM (#18144320)
With GraphQL, you are creating what is essentially a 1:1 mapping of the structure of the data you want to get back. It's a lot more intuitive than joins and such.
Re:SQL as an API language by eksemplar on Friday October 05, 2018 @01:32AM (#18145937)
A join in GraphQL looks like this:

{ getOrganisation(args){ name, employees{ Name, Position }}

(I added commas to make it easier to read, but they are not supposed to be there)

I know SQL, so I don’t think it’s necessarily easier, but what is easier, is non-sql-savvy people making efficient queries because GraphQL does a lot of the heavy lifting for you.