If that's dangerous in Firefox, isn't it just as dangerous, if not more, for them to reference other non-firefox files? I can't see this being a proper security measure for anything.

It is. The most dangerous part though is when you can access files in the browser cache because scripts in file:// can read files in file:// (so all your files) and you can really easily put files in the browser cache.

Firejail on GNU/Linux, Capsicum on FreeBSD etc do something for that, and IMO any tech-savvy users should use them.

Basically they restrict browser capabilities to see entire filesystem to the minimum extent possible so an webappp can't download from you your personal ssh settings, plain-text saved passwords in home, secret porn collection etc.