I'm slowly moving in that direction. Recently started using (pg-typed)[1] in our projects and its amazing, as it gives you the types from the database into typescript, and not the "general types from tables", but the exact specific types for each individual query.

Coupled with the same thing going the other direction where we get types from our api contracts (OpenAPI/Swagger) with (laminar)[2] means that our app is very close to the "if it compiles it will run" territory.

ORMs do give you a lot of convenience though. Things like "run this additional query every time you request this entity" thing for example like for logical delete, which is unpleasant to replicate in your database. But Postgres is so freaking powerful its more of the fact that we don't know how to do it properly than it not offering a good solution.

[1] https://github.com/adelsz/pgtyped [2] https://github.com/ovotech/laminar

The blog post[1] by the author about ORMs was what convinced me to purchase the book. I've had too many discussions with colleagues and tech friends struggling with N+1 problems and processing too much stuff in the application when they use mainstream ORMs to think it is a good idea. ORMs make the regular CRUD stuff simpler, but seem to make some more complex queries and transactions harder. There also seems to be an impedance mismatch between SQLs relational model and the OOP object relationship model.

Edit: I know there are ways to avoid N+1 problems with ORMs, but it seems to more easily sneak into code when your SQL queries look just like your application level code and you could easily enumerate over some SQL result, perform some action, and think that it builds an efficient query.

I've recently been working with a hobby project where I use the Clojure HoneySQL[2] library which essentially lets you build SQL queries as you normally would, but in Clojure's EDN syntax. It treats SQL queries as data. You can super easily evaluate them to get the resulting raw SQL query strings. There is no magic behind it and it encourages you to use the full power of your db.

[1] https://theartofpostgresql.com/blog/2019-09-the-r-in-orm/ [2] https://github.com/seancorfield/honeysql

Anyone who's interested in pgtyped may find this library comparison helpful: https://phiresky.github.io/blog/2020/sql-libs-for-typescript...

I haven’t read this yet but I love your take.

Far too often do I see developers doing analytics by slurping an entire table across the network and performing calculations on it in the application. Of course it appears to work in development with tens of kilobytes of records and a local database, but as soon as it’s deployed to production it unleashes chaos.

I consider myself very good at SQL but I do prefer to use ORMs for most things. However there’s no arguing that many of them have done a lot to obscure the incredible power inside a relational database, and perhaps go too far in hiding exactly when data is computed remotely vs. being pulled and operated on locally.

I’m very guilty of all those sins. Select all, then use array map, reduce and filter in the app code. I could draw a line in my life where I discovered window functions.

If the dataset is big enough, those operations actually do make sense because they can be translated to e.g. a Hadoop job - map / reduce / filter are all operations that don't depend on the sorting of a dataset, so in theory the dataset can be sharded across many machines.

Of course, that is a solution to a problem that people wish they had. Relational databases can run multi-million row queries in seconds, and then there's BigTable / BigQuery which scales SQL to improbable scales.

Exactly. I'm talking about the "select 5000 rows, serialize them to JSON, and send them over HTTP to the browser, only to render them in a table that never changes" type code.

One argument I've often heard is that since it's easier to scale your application compared to your database, pushing the calculations on your application is a way to get a better scalability in the long term. I wonder how true that is though.

I’ve seen applications go down hundreds of times from pulling entire tables locally to compute on them. I’ve seen maybe once where an application grew so successful that scaling the database became an extremely challenging topic.

And even in that case, I/O and contention were inevitably the problems. Not CPU.

I’ve similarly heard myths that you should be judicious when writing indexes because they can affect insertion performance. I’ve again seen hundreds of cases where under-indexing killed performance and zero where over-indexing caused problems.

99.9% of the time, you’re not the crazy special case. And if somehow you are, the solutions required are going to be nuanced and involve a ton of specific measurement. It’s widely unlikely you’ll accidentally avoid these problems through something like this.

That depends a lot on the workload, of course. But quite often the DB is not constrained on CPU, but rathen on IO or memory (or even networking), and calculations mostly consume CPU.

Moreover, calculations usually require just a small subset of columns in a table, and thus can use indexes efficiently, whereas grabbing all the columns to then filter them in application (because that's how many ORMs work, at least by default) becomes not only worse in terms of memory and networking, but also in terms of IO and CPU required on the DB side.

So overall I'm sceptical of that argument, unless there's clear proof from profiling that it's indeed the case.

Depends a lot. You need to consider the requirements more in detail, and especially if we are considering reads vs writes, and also what are the methods to write the data that needs scalable reads.

I have not used ORM, but used some basic SQL. Is ORM more performant? Why should I learn SQL if I can do all the things using ORM?

Quite the contrary, ORMs in general are less performant. Using ORMs boils down to two things, convenience and the ability to switch the underlying RDBMS. (For example all the OSS which says you can choose between MySQL, Postgres or SQLite.)

But if you support all RDBMS you only can support the smallest intersection between them and can't use advanced features like CTE, window functions, JSON support etc.

> But if you support all RDBMS you only can support the smallest intersection between them and can't use advanced features like CTE, window functions, JSON support etc.

That's not true at all, any more that it’s true of supporting all browsers with JS. You can use the advanced features where available, and implement logically (if not performance)-equivalent functionality using more basic functions where the advanced features aren’t available.

Or, if you are lazy, just have a reduced feature set available with less capable RDBMS engines. But, on any case, its simply not the case that an ORM that supports engines of varying capacity is limited to using only the least-common-denominator feature set.

Yeah, sure, you can always write raw SQL queries, but that has nothing to do with your ORM, does it?

I was talking about ORMs in general, not about the programmer in particular. So yes, you are right that if you have enough engineering resources you can support everything. But most ORMs don't help you with this, so it is not a feature of ORMs. You can always bypass it though.

The comparison with JS doesn't fit realy well, because JS is the tool you have to use. You don't have to use ORMs, plain SQL works fine as well.

When GP says 'you can use the advanced features', 'you' refers to the person developing the ORM, not to a developer using the ORM.

So the ORM author can do the hard work of allowing advanced features to work across databases. And it's transparent to the developer using the ORM.

Except, how many ORMs do that?

I use Postgres window functions with Hibernate and Doctrine, plus lots of other Postgres-specific functionality.

The real benefit of the ORM to me are that:

* some query results are cached

* the "unit of work" pattern allows me to distribute changes to an entity and then commit it as a transaction.

Anyway, the ability to suddenly switch RDBMS only works out in practice if you actively maintain support for multiple engines.

Solving a programming problem is usually finding the right data structure, so in that sense is not surprising. Most of the time I find that the real work is simply performed by the DB, and it's not just persistence, it's mantaining state and managing communication in concurrent applications.

I found that book really useful too, and it led me towards leaning more heavily on Postgres. For a long time using Postgres I just treated it as a ‘dumb’ data store with an ORM in front of it, but learning about CTEs, window functions, user defined functions, views, extensions, etc. has made me reconsider where a lot of application logic should go

That book is heavily biased toward doing everything you can in the database and boasts it as the only true way. Which IME is hardly true for many type of applications and phases of development. Many times having more brains in your application code leads to a simpler less decoupled solution.

Definitely going to take a look at this. I'm one of the few traditional developers in a company that's got a lot more analytics and database people than application or web people. In such a situation the company is loathe to use ORMs and the old adage of "never do your business logic in the database" doesn't really apply in practice. I've managed to learn enough SQL to get by and fulfill the software's requirements out of sheer necessity and I've actually got to the point I prefer to use it over an ORM, but I'd love to have a more systematic education in the language.

I'm thinking now about buying this because of your comment, but I'm wondering if full edition is worth extra 50$ compared to standard

It really depends how much $50 is worth to you. The book is brilliant by itself, but the datasets and unit test examples are a really good tool for playing with. Maybe if you have some data from a real-world project you could use that instead.

That's one of the most expensive technical books I've ever seen.

Holy smokes, $100?!?! You’re not kidding!

I’m always willing to buy technical books. I think it’s valuable to have material from different authors because they each have different perspectives and styles. E.g. CLRS vs Sedgewick vs Skiena. I also like to support the authors. However I’ll take a hard pass at this one.

My ability to level up is rarely due to the quality of the material, it’s more a function of how much time and effort into studying and learning. Time is the limiting factor in almost everything, not learning material.

In other words, learning isn’t about choosing “book A” vs “book B” but rather studying any books vs scrolling through HN, watching YouTube, or any of the other million blackholes of time.

I look at it that I spent $100 on this book, and got a $20K payrise. At that order of magnitude, the difference between spending $10 and $100 didn't matter.

You didn’t get the pay raise because spent $100, you got the pay raise because you spent hours, days, months.

It’s time, not money, you are spending.

From that point of view the $100 for the book is even less relevant.

I've been looking on brushing up on my SQL lately and have been going through these resources.

- Practical SQL, No Starch Press. ($30)

https://nostarch.com/practicalSQL

- Use The Index Luke ($15)

https://use-the-index-luke.com/

- Database Systems Concepts & Design by Georgia Tech on Udacity (free)

https://www.udacity.com/course/database-systems-concepts-des...

I can easily pay the $100, I cannot easily find more time, especially when there are a bunch of other things I'm spending time on. At a lower price I would likely buy the book "just because".

They sound good, too. Be good to hear a review if you get any of them.

The eBook bundle is $49 which is perhaps a little higher than average but doesn’t seem unreasonable.

For a pdf that doesn't come with the data for exercises it's borderline insulting.

Oh wow, used to work with Kris Jenkins but had no idea he was a SQL-guru, I always went to someone else on the team with queries!

He's a comedic genius too!