News for hackers. Stuff that matters.
Re:Debian's /tmpest in a teapot by candiddevmike on Tuesday June 04, 2024 @04:48PM (#40578768)
/var/tmp, IMO, shouldn't exist--the idea of temporary files that should persist beyond a reboot is tech debt, those files should exist under a proper /var/{cache,lib} directory. Podman uses this directory and it drives me nuts.

The XDG base directory spec (https://specifications.freedesktop.org/basedir-spec/basedir-...) was supposed to solve all of this...

Re:Debian's /tmpest in a teapot by NekkoDroid on Tuesday June 04, 2024 @06:20PM (#40579552)
/tmp/ is the ram-backed version for small files, while /var/tmp/ is the disk-backed version for large files and I also think both should be clean at bootup.

In my personally opinion the reaping period is way too long, especially for /tmp/ as only few people really have an actual uptime that long but it does make somewhat sense for /var/tmp/ as it currently is "persistent".

Re:Debian's /tmpest in a teapot by hda111 on Wednesday June 05, 2024 @02:38AM (#40582002)
Depends on the distro. In Debian /tmp is backed by the disk.
Re:Debian's /tmpest in a teapot by bayindirh on Wednesday June 05, 2024 @02:45AM (#40582042)
> In Debian /tmp is backed by the disk.

In Debian /tmp was backed by the disk. Trixie will come with RAM based /tmp. Update is pushed.

Re:Debian's /tmpest in a teapot by jkaplowitz on Wednesday June 05, 2024 @05:18AM (#40582953)
The article we’re discussing is exactly about Debian changing that for the next release. (The only thing that will be changing in this respect is the default, and possibly the configuration at the moment of upgrade to the next release. Debian has long allowed users to configure both RAM-backed and disk-backed /tmp, and both will remain possible.)
Re:Debian's /tmpest in a teapot by Terr_ on Wednesday June 05, 2024 @04:20AM (#40582609)
> I also think both should be clean at bootup.

I'm OK with a RAM-backed /tmp getting cleaned every boot simply because that's part of the technical tradeoff, however "on boot" is a bad trigger because its behavior is unreliable and can easily backfire and harm a user.

A classic example would be if the power is interrupted and the computer restarts. Unless the user is present and savvy enough to block it somehow, all of their short-term "throwaway" project/data just got deleted before they actually finished with it.

In other words, the difference to users between "I won't need this after tomorrow" versus "I won't need this by next boot" can be very important.

Re:Debian's /tmpest in a teapot by stuaxo on Wednesday June 05, 2024 @06:11AM (#40583257)
If the system went wonky because the disk was full, then clearing this on boot might help me get to a working system.
Re:Debian's /tmpest in a teapot by jraph on Wednesday June 05, 2024 @08:51AM (#40584202)
> Unless the user is present and savvy enough to block it somehow, all of their short-term "throwaway" project/data just got deleted before they actually finished with it.

That's on them.

I use /tmp like this, because I'm lazy and don't want to have to mess with a directory I need to routinely clean up, but I know that if the computer crashes, it's gone. That's on me.

If you don't like this risk, you need to work in persistent storage.

/tmp is for programs' temporary files. Manually using it like this is abusing it. That's fine, it doesn't hurt anyone beside the user doing it, but they need to bear the consequences of doing it.

Re:Debian's /tmpest in a teapot by Spivak on Wednesday June 05, 2024 @01:19PM (#40587698)
> If you don't like this risk, you need to work in persistent storage.

Which is literally what /var/tmp is and what the parent is advocating for.

However, clearing /tmp at boot is not the greatest technical choice. There's nothing really special about rebooting the system. The idea that files in /tmp are kept around for anywhere between minutes, hours, and days makes for bad policy which systemd-tmpfiles fixes quite elegantly. You shouldn't lose your temp file after 30 seconds because your machine lost power. You should be able to say "files created here are cleared after $short_time of inactivity."

Re:Debian's /tmpest in a teapot by semi on Wednesday June 05, 2024 @01:57PM (#40588162)
does inactivity mean writes? reads? stat() calls?

IMO setting the expectation of 'wont survive a reboot' is more consistent. but even that fails with private tmp files that could be tied to ephemeral units

Re:Debian's /tmpest in a teapot by Spivak on Wednesday June 05, 2024 @02:17PM (#40588393)
mtime atime ctime? If it's good enough for logrotate it's good enough for /tmp.
Re:Debian's /tmpest in a teapot by crabbone on Wednesday June 05, 2024 @09:28AM (#40584526)
> /tmp/ is the ram-backed version for small files

Small you say? The block size in that device is the size of the memory page: 4K by default. That's 8 times the very typical 512b for persistent storage.

I've been bitten many times when trying to start in-memory only VMs with a disk image that measured smaller than available memory only to run out of memory on boot :)

Let's say it's not really for small files. It's for few files. ;)

Re:Debian's /tmpest in a teapot by throw0101c on Tuesday June 04, 2024 @07:39PM (#40580028)
> is tech debt

Or it's useful system behaviour.

To-MAY-to, to-MAH-to.

> […] should exist under a proper /var/{cache,lib} directory.

Are these directories available to regular users like the tmp dirs are?

Re:Debian's /tmpest in a teapot by Smar on Tuesday June 04, 2024 @11:06PM (#40580990)
/home/.cache/ is for regular users.
Re:Debian's /tmpest in a teapot by thayne on Wednesday June 05, 2024 @12:00AM (#40581259)
System users often don't have a real home directory, but need somewhere to write files too.
Re:Debian's /tmpest in a teapot by karmarepellent on Wednesday June 05, 2024 @02:44AM (#40582040)
If feel like this is the moment where these discussions about old filesystem layouts start to go in circles.

Yes, system users tend to not have a home directory. But you are fully able to create one for a new system user if this is what you want to do.

You are also able to create all kinds of temporary directories and read-only filesystems for a process (e.g. using systemd) if this is what you want do to.

Yes there is a lot of fragmentation and no real standard, which is a little disappointing, but you sure have options to work with. And these options do not have to involve any legacy-ish filesystem components like /var/tmp that nobody knows the purpose of.

Re:Debian's /tmpest in a teapot by thayne on Wednesday June 05, 2024 @05:07PM (#40590648)
> filesystem components like /var/tmp that nobody knows the purpose of

The purpose of /var/tmp is temporary files that should survive a reboot. A potential example where that would make since is a partial download file. It is only needed temporarily but if a reboot happens while the download is in progress, you probably don't want to have to start over.

Also if /tmp is or might be a tmpfs, it is probably desirable to use /var/tmp for larger temporary files that don't fit in RAM.

Re:Debian's /tmpest in a teapot by dheera on Wednesday June 05, 2024 @01:48AM (#40581752)
and /home/.aws/, /home/.config/, /home/.docker/, /home/.dotnet/, /home/.dropbox/, /home/.mplayer/, /home/.vscode/, ...
Re:Debian's /tmpest in a teapot by numpad0 on Wednesday June 05, 2024 @08:13AM (#40583934)
disclaimer: IIUC, never experienced first hand.

/var was originally for network booting machines. / is read-only fileshare and /var is for user storage. /home -> /var/home too. It's campus wide workstations installation type of thing.

So, /var/tmp is kind of wrong, and so is /var/log/messages on a laptop, and besides the entire /var is useless on most systems. But it's not a huge waste of resources either, so it's left as it is along the rest of directories under / for years.

/var wasn't originally intended to be a system directory, it's just a variables directory. Almost by definition it's writable.

Re:Debian's /tmpest in a teapot by throw0101c on Wednesday June 05, 2024 @05:23PM (#40590795)
> disclaimer: IIUC, never experienced first hand.

I have run (Sun) workstations off of NFS, including completely diskless ones. /var was often done on a per-workstation basis.

(I'm actually planning out a diskless HPC cluster that will be deployed in the coming months.)

Re:Debian's /tmpest in a teapot by zbowling on Tuesday June 04, 2024 @05:07PM (#40578948)
Nah. XDG base directory was supposed to solve a ton of things 18 years ago, except none of them are hard-hitting user problems most folks really care about most of the time unless you are really anal about hidden folders in your home directory polluting things up (most people aren't), so it fails with partial support from apathy from some package devs on Linux to implement it.

Honestly, we should probably abandon the hope that we will ever get full universal adoption. It stinks even worse with the half-adoption state we are in now, with some apps writing to those directories and other apps not. If it wasn't for arch Linux folks trying to drive apps to adopt it more, I don't think it would be. Hell, default Debian/Ubuntu still doesn't set XDG_* directory ENVs by default (some flavors do), and some apps ignore the prescribed "if not set, use this default" nonsense in the spec and do what they have always done because it doesn't break compatibility for users.

Part of the spec stinks, too, like the spit between cache/config/data config where the spec has no written rules specified on when anything is expected to be cleaned up and when or what can or should be backed up by the user.

Let's move to containerizing most apps, putting everything in little jails so we don't have to deal with where they want to write to. They can have their own consistent and expected behaviors on their own island. Snapcraft, flatpack, or any of a bunch of other solutions are already available to solve this problem. Don't have to worry about what some app left behind or wrote to your system when it's all contained.

Re:Debian's /tmpest in a teapot by pseudalopex on Tuesday June 04, 2024 @07:21PM (#40579940)
> It stinks even worse with the half-adoption state we are in now, with some apps writing to those directories and other apps not.

It stinks about half as much as it used to.

> Part of the spec stinks, too, like the spit between cache/config/data config where the spec has no written rules specified on when anything is expected to be cleaned up and when or what can or should be backed up by the user.

Caches don't have to be backed up and can be deleted if you need space. Most people will want to back up configs. What other data should be backed up is a question different people will answer differently. And containers don't solve this.

Re:Debian's /tmpest in a teapot by immibis on Tuesday June 04, 2024 @10:46PM (#40580919)
I actually back up my Gradle cache in particular because things have a tendency to disappear from the internet sometimes, and permanently break builds of older stuff.
Re:Debian's /tmpest in a teapot by thesnide on Wednesday June 05, 2024 @02:11AM (#40581849)
That stuff that disappears, do you have means to patch and rebuild it?

Otherwise, you're directly heading for a nasty surprise at the worst moment.

Re:Debian's /tmpest in a teapot by immibis on Wednesday June 05, 2024 @11:19AM (#40585970)
The build dependencies are the problem. The build process downloads them from hardcoded internet URLs and they are cached for 30 days. By backing up the cache, I can (probably) rummage around it later to find anything that's been deleted from the internet.
[dead]
Re:Debian's /tmpest in a teapot by zbowling on Tuesday June 04, 2024 @05:18PM (#40579042)
Guess you have never used https://snapcraft.io/, https://flatpak.org/, or https://appimage.org/ because all of these do exactly that. Snap apps are straight linux containers and the others root like chroot jails I believe still.

Or even go take a look at Nix/NixOS and how they pull it off in another way. They have hermetic isolation down to a science.

Or heck, just look at what Android does, running each app under its own uid/gid, sandboxing 3rd party code, and keeping each app from reading and writing outside their little jails. Can't pollute a user directory or even write to /tmp if your user can't even enumerate it.

Hell, even built a whole sandboxing capability-based security model inside of Fuchsia at Google, which I worked on for 5+ years.

I've been building OSs for 20+ years, between Fuchsia and Android at Google and mobile/embedded products at Texas Instruments, so I hope I know what I'm talking about.

Re:Debian's /tmpest in a teapot by Brian_K_White on Tuesday June 04, 2024 @07:06PM (#40579845)
Android is a thoroughly managed appliance, not a general purpose OS.

Snaps should not exist. Flatpak and appimage should exist, but should not be used except in extreme cases.

You can have bad judgement for 20+ years easily.

Re:Debian's /tmpest in a teapot by zbowling on Tuesday June 04, 2024 @07:55PM (#40580122)
Nah. Your objections are rooted in the very limited definition of what an OS is and what a user application model is that fits it. There is no reason why each process/app can't be sandboxed. In fact, it should be for security (we did it in fuchsia). It's actually the way things work with apps from the app store on MacOS in a lot of ways where you can't escape your jail except through what is explicitly entitled.

General purpose just means that it's generally useful for a wide range of applications. Android is that. Hell, you can find Android running small appliance server infrastructure and powering IoT devices. Even in 2024, iOS/iPadOS is general purpose at this point, and they have VERY different application models from legacy app models you find on Windows and Linux. You wouldn't not call NixOS a general-purpose OS, and it's like flatpak for literally every process down to things like bash, grep, and vim.

Snaps are fine. Aversion is only from how it was introduced to people in Ubuntu, but conceptionally, is great. cgroups wrapping user processes to box them is not only good for security but also for stability and solving dependency versioning issues. It's brilliant. It's similar to what we did in Fuchsia at Google (we took it to another level because we had no legacy to deal with).

And sure, maybe I have bad judgment on some things. I contributed a ton to GNOME in the early 2000s both code and ideas that were horrible in hindsight, but I'm not still stuck in an outmoded mental model for thinking about my user environment.

Re:Debian's /tmpest in a teapot by fragmede on Tuesday June 04, 2024 @06:52PM (#40579765)
Or Qubes, which goes further and features per-app VMs. Snaps was foisted on the community which was then unwelcoming of it, but per-app isolation isn't the worst idea.
Re:Debian's /tmpest in a teapot by account42 on Wednesday June 05, 2024 @06:43AM (#40583427)
> Guess you have never used https://snapcraft.io/, https://flatpak.org/, or https://appimage.org/ because all of these do exactly that. Snap apps are straight linux containers and the others root like chroot jails I believe still.

AppImages are not containers at all. They bundle up the program data into a single archive but do not do any sandboxing and leave programms to write their user/config/cache files to wherever they would be written without AppImages, i.e. in xdg-basedir locations. As it should be.

> Or even go take a look at Nix/NixOS and how they pull it off in another way. They have hermetic isolation down to a science.

NixOS's packaging is also completely orthagonal to the xdg-basedir spec.

> I've been building OSs for 20+ years, between Fuchsia and Android at Google and mobile/embedded products at Texas Instruments, so I hope I know what I'm talking about.

None of those are desktop operating systems. Please stay away from those with your anti-user opinions.

Re:Debian's /tmpest in a teapot by CoastalCoder on Tuesday June 04, 2024 @07:16PM (#40579911)
> I was sort of with you until this, when I realized you have no idea what you are talking about.

I love the pith of your comment, but it wasn't very nice.

Like the Churchill quip about being called drunk.

Re:Debian's /tmpest in a teapot by cwillu on Tuesday June 04, 2024 @05:29PM (#40579141)
Yeah, I had upvoted the comment, and retracted the upvote when I got to the last sentence. Nobody who has anything kind to say about snaps has any business anywhere near my desktop.
Re:Debian's /tmpest in a teapot by Wowfunhappy on Tuesday June 04, 2024 @09:45PM (#40580660)
> unless you are really anal about hidden folders in your home directory polluting things up

n = 1, but, uh, this is one major reason I don't use Linux. It really bugs me.

Re:Debian's /tmpest in a teapot by BenjiWiebe on Wednesday June 05, 2024 @01:52AM (#40581768)
If hidden pollution bothers you, and you use Windows, don't browse the registry. shudder

Actually AppData can get a bit polluted too.

Re:Debian's /tmpest in a teapot by thesnide on Wednesday June 05, 2024 @02:09AM (#40581838)
Parent seems to care more about 'not polluting something i see all the time'.

Typical of our society, we want to make everything someone else's problem.

I guess we deserve some of the SaaS enshittification and it's spending obesity ;-)

Re:Debian's /tmpest in a teapot by Wowfunhappy on Wednesday June 05, 2024 @06:58AM (#40583505)
> Parent seems to care more about 'not polluting something i see all the time'.

Yes, this.

> Typical of our society, we want to make everything someone else's problem.

I don't think that's a fair analogy.

It's more like moving junk from the living room to the attic. I'd prefer to have a clean living room and a messy attic.

Re:Debian's /tmpest in a teapot by vetinari on Wednesday June 05, 2024 @08:04AM (#40583866)
>> Parent seems to care more about 'not polluting something i see all the time'.

> Yes, this.

Like "My Documents" (or "Documents" nowadays)? Now that's some dumping ground.

Re:Debian's /tmpest in a teapot by Wowfunhappy on Wednesday June 05, 2024 @09:10AM (#40584390)
That's bad too, but at least it's not right in my home folder! When I have to use Windows, I basically just avoid using Documents for files. (But my daily driver is an ancient version of OS X, which has ~/Library for these things.)
Re:Debian's /tmpest in a teapot by vetinari on Wednesday June 05, 2024 @10:23AM (#40585143)
Oh, mac has the same issue and for the same reason: the same programs will create the same garbage.

~/Library also isn't something that is used to be; it was good idea, but mostly ignored. Basically similar idea as XDG_(CONFIG|CACHE|DATA)_DIR.

What's worst, is that many projects ignore any effort to move them to XDG schema. They take the approach, that they always did it this way, it works, so why changing it.

Dot files in my homedir on Sonoma machine:

    .CFUserTextEncoding
    .DS_Store
    .Trash/
    .Xauthority
    .amt
    .android/
    .anyconnect
    .bash_history
    .cache/
    .cargo/
    .config/
    .cups/
    .degit/
    .deno/
    .docker/
    .eclipse/
    .emacs.d/
    .gitconfig
    .gitignore_global
    .grass7/
    .hex/
    .homeassistant/
    .ipython/
    .java.login.config
    .jupyter/
    .kube/
    .lemminx/
    .lesshst
    .local/
    .matplotlib/
    .mix/
    .mume/
    .node/
    .notable.json
    .npm/
    .nvm/
    .oh-my-zsh/
    .oracle_jre_usage/
    .pip/
    .profile
    .psql_history
    .python_history
    .rest-client/
    .rustup/
    .sqitch/
    .ssh/
    .subversion/
    .swiftpm/
    .thumbnails/
    .viminfo
    .vscode/
    .wget-hsts
    .wine/
    .yarn/
    .yarnrc
    .zcompdump
    .zsh_history
    .zsh_sessions/
    .zshenv
    .zshrc
Re:Debian's /tmpest in a teapot by tristan957 on Wednesday June 05, 2024 @03:42PM (#40589572)
Many of these you can move if you wanted. For instance, git looks in $XDG_CONFIG_HOME.
Re:Debian's /tmpest in a teapot by Wowfunhappy on Wednesday June 05, 2024 @09:21PM (#40592446)
My home directory is much cleaner because I've made a concerted effort to keep it that way, in a couple of cases even recompiling software. I found this much more difficult on Linux.
Re:Debian's /tmpest in a teapot by account42 on Wednesday June 05, 2024 @06:38AM (#40583403)
> Hell, default Debian/Ubuntu still doesn't set XDG_* directory ENVs by default (some flavors do)

You are supposed to leave them unset if they match the defaults.

Programs ignoring the spec is another matter and should be fixed in those programs not by needlessly bloating the environment of every process with meaningless data.

Re:Debian's /tmpest in a teapot by zbowling on Wednesday June 05, 2024 @01:50PM (#40588062)
> meaningless data

Things like .git and .ssh not are not meaningless data even though they are not in ~/.config/ssh/ or ~/.local/shared/git.

Re:Debian's /tmpest in a teapot by pjmlp on Wednesday June 05, 2024 @06:52AM (#40583474)
As someone that knows Linux since Slackware 2.0 in 1995, the adaption of XDG has been quite flexible across the Linux/BSD landscape.
Re:Debian's /tmpest in a teapot by belorn on Wednesday June 05, 2024 @08:20AM (#40583983)
They could merge /var/tmp and /var/cache, with the intention to discourage future use of /var/tmp. With users, programs and admins already using /var/tmp as a persistent cache, a name change seems like the best solution.
Re:Debian's /tmpest in a teapot by tremon on Wednesday June 05, 2024 @06:31PM (#40591379)
Actually, no. The XDG specifications are intended for desktop environments, the focus is on end users throughout the specs. System services and their locations are out of scope. The standard for filesystem layout was the Filesystem Hierarchy Standard [0], which predates the XDG by about a decade. Although Debian-before-multiarch was the distribution that most closely adhered to the FHS, it was not a Debian-only document.

That being said, all my systems have /var/tmp as just a symlink to /tmp, regardless of what the FHS may have to say about it -- and it's cleaned weekly, not just on reboot.

[0] https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch05s15.htm... is the current home, apparently