News for hackers. Stuff that matters.
Re:Filesystem and Database are not cutting the problem space... by kazinator on Friday November 27, 2020 @03:15AM (#25226517)
This is wrong; you can implement atomic transactions in filesystems. As in prepare a change, and then publish it, such that the consumers of that event only see the before and after view, and not something in between. There are many ways to do this, on various scales, involving parts of a file, a whole file or multiple files. Filesystems really have no such disadvantage over databases.

Think about a remote git repo. Multiple users can push. It's based on files.

This HN site is based on files.

Re:Filesystem and Database are not cutting the problem space... by skissane on Friday November 27, 2020 @06:12AM (#25227670)
Common filesystems provide some atomic operations; for example, on POSIX systems, rename() is atomic for local filesystems (but not necessarily remote ones)

There are also some filesystems which expose special transactional features beyond what common filesystems offer. For example, NTFS has filesystem transactions, although it was rarely used and Microsoft has deprecated it – part of the reason for its rare use, is that with Win32 it isn't seamless to adopt, all existing filesystem calls have to be replaced with *Transacted equivalents. (Apparently the undocumented NT level API is more seamless, but well, it's undocumented.)

Re:Filesystem and Database are not cutting the problem space... by zupa-hu on Friday November 27, 2020 @07:09AM (#25228004)
Hi, author here. Sure, multiple users can do a git push but don't neglect that git is also offering a higher level abstraction, basically a storage system. That's no different from running any other storage system, like a filesystem, except you name it differently.

There are also other hacks to go around it but none of them is a simple solution abstracting away the complexity. Not a clean filesystem that a novice developer could start using without messing it up. They all greatly increase mental complexity.

Re:Filesystem and Database are not cutting the problem space... by lookdangerous on Friday November 27, 2020 @01:18AM (#25225967)
Maybe it’s because it’s late, but I don’t really follow what this is. It seems cool, but the explanation is too abstract
Re:Filesystem and Database are not cutting the problem space... by zupa-hu on Friday November 27, 2020 @06:52AM (#25227887)
Hey, author here, thanks for the feedback. I've been working on this problem space for a while and decided to write it all out one problem at a time. Yeah this one was quite heavy on the theory department but I plan to follow it up with a demo on how to actually use it. That should be more hands on. The problem was, I felt like I would be immediately asked on the theory behind it, plus there were people already interested so I thought it's better to have it explained. I agree with your point though.
Re:Filesystem and Database are not cutting the problem space... by bleepblorp on Friday November 27, 2020 @12:01AM (#25225670)
This post is advertising (spam) and contains no technical details.
Re:Filesystem and Database are not cutting the problem space... by franciscop on Friday November 27, 2020 @12:29AM (#25225753)
Non-technical blog posts are fine in HN as long as it appeals to a technical audience.

That said, this is definitely an ad where the only possible solution for the problems explained is the company product, flagged.

Re:Filesystem and Database are not cutting the problem space... by wyattpeak on Friday November 27, 2020 @01:24AM (#25225985)
This seems like an unreasonable criticism. Seeing a problem without a good solution and creating a solution to it seems to me the very essence of hacking.
Re:Filesystem and Database are not cutting the problem space... by franciscop on Friday November 27, 2020 @05:31AM (#25227375)
I would agree with that if this was a "explain how we fixed X" style of blog post. It instead starts/reads like a general problem domain informative post, but then it only proposes their solution. If the title was e.g. "How Boomla fixes the database/filesystem problem" or similar that'd be more fair.
Re:Filesystem and Database are not cutting the problem space... by monocasa on Thursday November 26, 2020 @11:22PM (#25225527)
Reminds me of the ancient ways of the hierarchical DBs like IBM's IMS.
Re:Filesystem and Database are not cutting the problem space... by zupa-hu on Friday November 27, 2020 @06:54AM (#25227899)
Author here. Interesting, never heard of that! Thanks for mentioning!
Re:Filesystem and Database are not cutting the problem space... by mikequinlan on Thursday November 26, 2020 @11:32PM (#25225558)
IMS, not ISM.
Re:Filesystem and Database are not cutting the problem space... by monocasa on Friday November 27, 2020 @01:15AM (#25225945)
Thanks, I'm dyslexic and mess stuff like that up regularly.
Re:Filesystem and Database are not cutting the problem space... by Brian_K_White on Friday November 27, 2020 @04:51AM (#25227093)
If this problem is as real and fundamental as presented, and the solution really is this equally fundamental new paradigm, then it should be a kernel module and not a product.
Re:Filesystem and Database are not cutting the problem space... by zupa-hu on Friday November 27, 2020 @07:03AM (#25227963)
The problem is, it's not enough to have a transactional filesystem as it will inevitably have a different API, one that is not POSIX compliant. Also, you can't jut make the filesystem transactional, in the end you also need an entire transactional OS built around it. Some years ago I set out to eliminate the accidental complexity and kept removing parts of the system that caused any specific problem. In the end, the filesystem + OS turned out to be the biggest culprit.