News for hackers. Stuff that matters.
Re:Stop building databases by roenxi on Friday December 01, 2023 @05:16PM (#38493260)
There is an interaction here between the "what gets measured gets managed" principle and the sunk cost fallacy.

The problem with databases is actually complexity. Any individual feature is more or less safe, but around the time reliability, caching and indexes get matched together there is a complexity explosion and it doesn't (normally, anyhow) make sense to implement a domain-specific DB (call is a DSD?).

But, around the time a company has invested in implementing those 3 features and discovered that it has sunk a lot of resources into the DSD, is politically averse to recommending it be stripped out and there is a high real cost to taking out the tech debt in one go.

Really the problem here is SQL's syntax. If using a basic relational database was a pleasant experience that involved some familiar C-like syntax instead of broken English people would be more tempted to go with a DB instead of rolling their own. The NoSQL databases were a good step in that direction, but then they by and large overfocused on big data instead of everyday usefulness. Things like Redis took hold which is nice.

Making it easy to run SQL is a reasonable approach, but the problem is that the good databases - I like postgres - are SQL native and it is hard to get efficiency without speaking the DB's language. We really need a PostgresPostSQL database that is a perfect postgres clone but primary parser supports a language with good syntax.

Re:Stop building databases by totalhack on Friday December 01, 2023 @07:10PM (#38494373)
Have you ever looked at adding a semantic layer on top of a db for those that prefer to avoid direct SQL?
Re:Stop building databases by lmm on Saturday December 02, 2023 @03:52AM (#38497142)
You'd need databases that were open to splitting out and layering like that, and I don't think any major SQL databases are. (I mean, most SQL databases are still written in C-family languages; splitting out the PostgreSQL parser into its own library took 20 years and still hasn't really been done properly, they do some weird sync thing from the mainline postgresql codebase rather than being the actual trunk for parser development and having postgresql proper use it as a dependency). Maybe you could do something with H2?
Re:Stop building databases by holoduke on Saturday December 02, 2023 @03:02AM (#38496864)
What is exactly hard about sql? Every dev imho should know it. And sql syntax is good and proven too be long lasting. Maybe investing some time in actually learning it instead of bashing it will help you further.
Re:Stop building databases by lmm on Saturday December 02, 2023 @03:48AM (#38497125)
> What is exactly hard about sql?

- No non-nullable types (at the expression level). No way to express e.g. normal boolean logic

- No real data structures (at the expression level), unless you count rows, which are not first-class values. Even collections aren't first-class

- Very awkward control flow constructs. E.g. look at how you write recursive queries. Even if/else is weird. It's a classical Turing Tarpit: everything is possible but nothing is easy

- Absurdly complex grammar. Something like 200+ keywords. Bizarre and inconsistent quoting rules

- Weird rules about where expressions can be used, and where you have to put variables. Often if you want to extract out a common subexpression and reuse it you have to completely restructure your statement.

- Missing basic functionality, e.g. no string concatenation operator (except in some nonstandard dialects)

- No libraries. No standard way of reusing code. No unit testing framework, which means in practice most projects have no testing.

It's a terrible language by any reasonable standard. Its closest cousin is COBOL, and that's honestly less awful to write.

Re:Stop building databases by holoduke on Saturday December 02, 2023 @08:27AM (#38498475)
If you run into issues with non nullable types and missing of real data structures then i guess you dont get the purpose of SQL and its tech. Its a query language to retreive data. thats the only purpose. With under the hood a crafty machine that enables atomic principles and depending on the sql tech other different functionalities like scalability and more.. No Libraries? There are a zillion battle tested libs out there. Some are used by companies like Paypal, SpaceX, Tiktok, youtube, Gmail and many more. And why you would unit test datasets in a database? You normally use mocks to test against your data layer. Most if not all software stacks have mock libs available.
Re:Stop building databases by lmm on Saturday December 02, 2023 @06:04PM (#38503101)
> If you run into issues with non nullable types and missing of real data structures then i guess you dont get the purpose of SQL and its tech. Its a query language to retreive data. thats the only purpose.

That's no excuse for making it bad.

> No Libraries? There are a zillion battle tested libs out there.

Not in SQL.

> And why you would unit test datasets in a database?

For the same reason as anywhere else.

> Most if not all software stacks have mock libs available.

Exactly. They're a basic piece of functionality you'd expect most languages to provide, but not SQL, because it's awful.

Re:Stop building databases by paulryanrogers on Saturday December 02, 2023 @09:50AM (#38498959)
> no string concatenation operator

`||` is the SQL standard. IMO its biggest deficiency is a single NULL nullifies the whole thing. But at least that's consistent with how NULL behaves in many other cases.

Re:Stop building databases by runeks on Sunday December 03, 2023 @11:01AM (#38508025)
- No custom sum types

I would love to be able to say e.g.: this column is either a PK in the “results” table or a PK in the “errors” table.

Re:Stop building databases by LectronPusher on Monday December 04, 2023 @01:29AM (#38514267)
What are your thoughts on the prql language?
Re:Stop building databases by lmm on Thursday December 07, 2023 @05:40PM (#38562792)
Sounds promising. I haven't had the chance to actually use it though.
Re:Stop building databases by jakubmazanec on Saturday December 02, 2023 @02:11PM (#38501095)
EdgeDB authors explain problems with SQL quite nicely: https://www.edgedb.com/blog/we-can-do-better-than-sql
Re:Stop building databases by GuB-42 on Saturday December 02, 2023 @09:14AM (#38498735)
SQL often gets criticized, and I think for good reasons, but why didn't we come up with something better?

We have dozens of programming languages in use for general programming, it is a field that is constantly evolving. Even JS, which is hard to change because that's what browsers run and you don't control your client's browser is seeing some evolution, using transpilers, and now WebAssembly.

But for databases, there is only one, and that SQL. There are some alternatives, but no one comes close to SQL in terms of usage. So maybe SQL is not that bad after all.

Maybe the reason is that the relational model is really good, and attempts to deviate from this will only work in a niche. That a declarative style is also really good, and again, you won't have much success if you deviate from this. And if you end up doing SQL with a different syntax, for most people, it will be too small of an improvement to justify changing their way.

Re:Stop building databases by runeks on Sunday December 03, 2023 @10:58AM (#38508001)
> Really the problem here is SQL's syntax. If using a basic relational database was a pleasant experience that involved some familiar C-like syntax instead of broken English people would be more tempted to go with a DB instead of rolling their own.

I wish Postgres had a stable, more low-level API than SQL. Perhaps something that resembles the query plan I get from EXPLAIN. An application written against this API could implement an SQL DB: (1) parse SQL (2) implement a query planner that spits out a query plan written against this API.