News for hackers. Stuff that matters.
Re:Scaling Databases at Activision [pdf] by vecter on Friday April 21, 2023 @04:36PM (#35659714)
Sorry, assume I'm dense.

> Take a well-formed 3NF schema and disable foreign key constraints.

I'm familiar with 3NF, but can you expand on what this means and how it relates to foreign keys? Or just point me to an article, I don't want to waste your time. I did some googling but wasn't sure where to proceed from your post.

Re:Scaling Databases at Activision [pdf] by berkle4455 on Friday April 21, 2023 @05:36PM (#35660538)
Say you have a super basic setup:

  user = {user_id, email}
  order = {order_id, user_id}
order.user_id is a foreign key to user.user_id. That's a perfectly valid and reasonable way to organize things.

Enabling RDBMS-enforced foreign key constraints is the issue. It slows everything down dramatically.

Re:Scaling Databases at Activision [pdf] by vecter on Friday April 21, 2023 @05:44PM (#35660624)
I see, thanks. Basically just store the foreign keys yourself as columns in relevant tables and perform the joins in SQL without having the DB enforce FK integrity with every insertion/update/delete. Would it be fair to say that read speeds are unaffected by this?
Re:Scaling Databases at Activision [pdf] by berkle4455 on Friday April 21, 2023 @05:53PM (#35660729)
Yes. Quite literally the only difference is not enabling foreign key constraints. Read speeds unaffected correct.
Re:Scaling Databases at Activision [pdf] by kgeist on Saturday April 22, 2023 @04:47AM (#35664551)
>It slows everything down dramatically

How slower is it really? MySQL automically creates indexes for foreign keys so I don't think it slows down "dramatically", just an additional indexed retrieval?

Do you know of any benchmarks which show the difference?

Re:Scaling Databases at Activision [pdf] by Philip-J-Fry on Friday April 21, 2023 @05:46PM (#35660653)
They're not saying 3NF enables you to remove foreign keys. They are talking about removing foreign key constraints from your RDBMS of choice.

Something like SQL Server can enforce foreign key constraints if you explicitly tell it your relationships between tables. The downside is that having this referential integrity costs you performance as the database has to check your relations when inserting/updating/deleting rows. E.g. checking that a foreign key is pointing at a valid primary key, checking that you aren't leaving invalid foreign keys when deleting a primary key, etc. This is to prevent you inserting bad data into the database.

You can delete these constraints and still have the exact same behavior so long as your code is correct. It just means that the database isn't going to stop you writing bad data.

Re:Scaling Databases at Activision [pdf] by vecter on Friday April 21, 2023 @05:54PM (#35660747)
That makes sense. For workloads where write performance isn't very important but read performance is, this sounds like it may still be a worthwhile tradeoff to have that extra level of data integrity.

My sense is that many typical CRUD apps aren't writing gargantuan volumes of data or making very complex edits, and if they do, it's ok if it takes a second longer. Usually read speed is more of a bottleneck for user-facing applications, but I'm sure there are probably some examples where this tradeoff is worth it.

Re:Scaling Databases at Activision [pdf] by AlisdairO on Friday April 21, 2023 @07:23PM (#35661570)
> so long as your code is correct

This is a pretty tough definition of correct, though. Without foreign key constraints you'll have a really tough time dealing with concurrency artifacts without raising your isolation levels, which generally brings larger performance concerns.

Re:Scaling Databases at Activision [pdf] by remram on Saturday April 22, 2023 @09:23AM (#35666372)
Why do you think that?

My experience is that if you have a moderate amount of foreign keys, a lot of DBMS (not Postgres) will refuse the `ON DELETE CASCADE` (in the diamond case), and you have to do it "manually" anyway (from your query builder).

Re:Scaling Databases at Activision [pdf] by AlisdairO on Saturday April 22, 2023 @11:10AM (#35667339)
I think it because a significant fraction of my career has been spent fixing db-concurrency-related mistakes for people once they hit scale :-).

I’m not talking about using cascade - this applies perfectly well to use of on delete restrict. FKs are more or less the only standard way to reliably keep relationships between tables correct without raising up the isolation level (at least, in most dbs) or doing explicit locking schemes that would be slower than the implicit locking that foreign keys perform.