News for hackers. Stuff that matters.
Re:PostgREST by fogetti on Sunday November 03, 2019 @08:50PM (#21438307)
Inherent architectural flaws: - Complete lack of separation of concerns: you can see this by the terrible idea of simply slapping a transformation procedure to a non-specified part of the codebase. Every time you do this you end up having to maintain a codebase which has no architecturally defined place anywhere. See: https://postgrest.org/en/v6.0/how-tos/casting-type-to-custom...

- There is no ORM that you can control. It doesn't mean that there is no transformation needed though. It just simply means that you lose control over it. All the work which were once under your control now is handled by the database layer. Of course you can go to that layer and change the logic of the framework yourself (by sending strangely formatted HTTP requests according to the doc), but then you gain nothing by using this framework. Of course this problem stems from the same fact that there are no separation of concerns. So basically the framework is simply ignoring the fact that there is an inherent object relational impedance mismatch. Of course the authors are aware that there is a mismatch but they simply sweep it under the rug by saying that:

  "The PostgREST URL grammar limits the kinds of queries clients can perform. It prevents arbitrary, potentially poorly constructed and slow client queries. It’s good for quality of service, but means database administrators must create custom views and stored procedures to provide richer endpoints."
This will come back and bite you in the ass when you try to work with anything slightly complicated like inheritance for example. So the authors basically dump every single person in the world under the category of poor programmers, and they offer their excellence in place of your lack of skill or talent. Go, figure! I am always very skeptical when I read something like this. See: https://postgrest.org/en/v6.0/api.html#resource-embedding

https://postgrest.org/en/v6.0/api.html#custom-queries

- No testability: pl/pgSQL is a really poor language to write unit/integration tests for, mostly because it's ancient and it's procedural and there are no native frameworks to do so. Of course you can use pl/Python and others, but then again, why would you when you can write the same thing in Python without embedding the code in Postgres, and the Python ORM would take care of having the same performance on the database level for you.

- Lack of tools for migrations, etc. The list goes on.

Re:PostgREST by steve-chavez on Sunday November 03, 2019 @11:14PM (#21438990)
Coming from a traditional layered approach I can see why this "de-layered" approach can seem wrong. After all I had to ditch the knowledge I had about Django ORM, Sequelize, HIbernate(I wish I could forget entirely about Spring/JPA books I've read but I still remember some parts). But as mentioned in other comments if you give it 5 minutes—and embrace the db—perhaps you'd see the advantages of this approach over more traditional ones.

> There is no ORM that you can control

You don't need an ORM to control your database. If you treat the database as the source of truth(this would be the architecturally defined place) you can do data transformations with plain and short SQL. The great thing about this is that you can leverage PostgreSQL rich functionality and extension ecosystem and you don't have to worry if your ORM supports a pg type or if you need a plugin for that.

> by sending strangely formatted HTTP requests according to the doc

The "strangely formattted request" wouldn't be so strange if you note that PostgreSQL cast operator is "::"(e.g. `select col::json from table`).

> Python ORM would take care of having the same performance on the database level for you.

That's just false. ORMs have to point to the lowest common denominator of databases. They don't use the full potential of pg(things like LATERAL joins, json_agg, etc). PostgREST queries are specifically tailored for pg and they're being constantly improved.

> No testability

I've shared this before, but by using pgTAP[1] you can get native SQL tests.

HNers have shared migration tools in other comments.

[1]: https://pgtap.org/

Re:PostgREST by AdieuToLogic on Monday November 04, 2019 @12:37AM (#21439282)
> Coming from a traditional layered approach I can see why this "de-layered" approach can seem wrong.

One of the most significant architectural defects of this approach is by demanding the persistent store provide functionality beyond solely managing data, it both conflates disparate concerns (business logic + data management) as well as unduly complicates system evolution.

> If you treat the database as the source of truth(this would be the architecturally defined place) you can do data transformations with plain and short SQL.

And if you treat the database as the source of truth for managing data, then the idea of making it an application engine would be anathema.

Re:PostgREST by steve-chavez on Monday November 04, 2019 @01:00PM (#21444333)
> disparate concerns (business logic + data management)

We argue that all "business logic" is in fact "data logic" and they're the same concern. There are several comments on this post on how to evolve this approach(only use VIEWs/sps in a single schema and expose this to clients).

> And if you treat the database as the source of truth for managing data, then the idea of making it an application engine would be anathema.

Why? Could you elaborate more on that?

Re:PostgREST by AdieuToLogic on Saturday November 09, 2019 @11:32PM (#21496410)
> > And if you treat the database as the source of truth for managing data, then the idea of making it an application engine would be anathema.

> Why? Could you elaborate more on that?

Sure.

Your previous declaration of:

> We argue that all "business logic" is in fact "data logic" and they're the same concern.

Is both the source of my position and your question.

For at least the last 50 years of which I am aware, the idea of "data rules all" has inextricably lead to migrating system logic closer to where the data resides. In the last 35-ish years, this meant defining application logic in stored procedures executed in an RDBMS (such as what you describe).

This architectural philosophy is commonly know as "client/server", or "two-tier systems," and has been studied extensively (as I am sure you are aware). What makes this relevant to the argument found in the "Motivation" section on the PostgREST site is:

  Using PostgREST is an alternative to
  manual CRUD programming.
And then:

  The PostgREST philosophy establishes
  a single declarative source of truth:
  the data itself.
To achieve that, the logic must reside within the RDBMS in order to fulfill the project goal set forth in the very first line of the site's home page:

  PostgREST is a standalone web server
  that turns your PostgreSQL database
  directly into a RESTful API.
For all intents and purposes, PostgREST could be considered an RDBMS connection pool, with the connection protocol being HTTP. Note that this is not a judgement, only an observation.

Now, consider a different system architecture commonly known as "three-tier" or "multi-tier." The philosophy of this type of system is largely defined by separation of concerns. This is to say, have each tier be responsible for a single "layer" involved in system definition.

In multi-tier systems, the persistent store (RDBMS) is solely responsible for ensuring data integrity. It may, in fact, service disparate components as well as disparate components may use more than one persistent store. An example of the latter is PostgreSQL and Redis deployed in concert. Programs, components, and/or systems which interact with a persistent store are responsible for providing the behaviour expected by their clients whilst the persistent store is expected to enforce rules regarding the relationships it manages.

Since the persistent store is only asked to fulfill a data integrity contract, having it also attempt to be an application engine would clearly conflate concerns, limit what persistent stores could be used, as well as result in assumptive implementations. Hence anathema to those who would treat the persistent store (database) as the source of truth strictly for the one concern which it was designed to address:

Managing data.

Re:PostgREST by steve-chavez on Sunday November 10, 2019 @05:18PM (#21500544)
I see what you mean. This is the view that considers that the database should only be a Persistence Layer and not a Processing Engine.

There is a great talk[1] that @felixyz shared regarding the shortcomings of the Persistence Layer approach and the advantages of Processing Engine approach(which we at PostgREST embrace).

In summary, having the "business logic" away from the db results in worse performance: more network roundtrips, ineffective cache utilization, wasted CPU cycles in traversing your programming language VM(plus all libs) code paths. Turns out this actually puts more strain on the DB and makes your whole system scale worse.

> Hence anathema to those who would treat the persistent store (database) as the source of truth strictly for the one concern which it was designed to address: Managing data.

I think marrying these concerns and having them in a single place is more of a benediction for OLTP systems development and maintenance. After all, PostgreSQL has gotten more powerful features over the years and it has evolved to do more than data management(LISTEN/NOTIFY, JIT, etc).

I really recommend watching that talk. Btw, slides here[2].

[1]: http://www.prohuddle.com/webinars/ToonKoppelaars/ThickDB.php

[2]: https://www.nloug.nl/downloads/ogh20170615_toon_koppelaars.p...

Re:PostgREST by fogetti on Sunday November 03, 2019 @11:35PM (#21439073)
This is still a horrible idea.

No IDE support, no debugging support, poor language choice. It's just simply screaming NO NO NO NO NO NO NO on every level of the framework.

Not to mention that these days it is more important than ever to make the code also executable in cloud environments like GCP Cloud SQL or AWS RDS.

> You don't need an ORM to control your database.

You are actually wrong about that and you are basically the living proof that I was right. You didn't even try to address what I was writing about object relational impedance mismatch and inheritance.

> you can leverage PostgreSQL rich functionality

I think this is a slightly outdated world-view. As I mentioned above these days the most important driving factors are not what a specific DB vendor supports but mostly what the cloud providers do. And many of the cool things are only supported by plugins in Postgres anyway.

> you can do data transformations with plain and short SQL

You guess what? You can do the same in ORMs too! In any language which has a Postgres DB driver! No one is stopping you from that.

Re:PostgREST by steve-chavez on Sunday November 03, 2019 @11:57PM (#21439148)
> object relational impedance mismatch and inheritance

Can you elaborate on that? PostgreSQL is an object-relational dbms btw, it supports inheritance[1]. Though I don't know exactly what impedance are you referring to since PostgREST doesn't use an ORM.

[1]: https://www.postgresql.org/docs/12/tutorial-inheritance.html