News for hackers. Stuff that matters.
Re:Running Out of Disk Space in Production by layer8 on Tuesday April 07, 2026 @09:35AM (#47675165)
Better fill those files with random bytes, to ensure the filesystem doesn’t apply some “I don’t actually have to store all-zero blocks” sparse-file optimization. To my knowledge no non-compressing file system currently does this, but who knows about the future.
Re:Running Out of Disk Space in Production by freedomben on Tuesday April 07, 2026 @10:12AM (#47675699)
Yep, btrfs will happily do this to you. I verified it the hard way
Re:Running Out of Disk Space in Production by kccqzy on Tuesday April 07, 2026 @10:27AM (#47675935)
Well btrfs supports compression so that’s understandable. However I personally prefer to control compression manually so it only compresses files marked by me for compression using chattr(1).
Re:Running Out of Disk Space in Production by freedomben on Tuesday April 07, 2026 @02:03PM (#47679041)
I've switched to that also. It surely wastes some space but being able to reason about file space is worth it to me for now
Re:Running Out of Disk Space in Production by ape4 on Tuesday April 07, 2026 @09:49AM (#47675361)
If I recall correctly:

    dd if=/dev/urandom of=/home/myrandomfile bs=1 count=N
Re:Running Out of Disk Space in Production by fragmede on Tuesday April 07, 2026 @12:05PM (#47677443)
bs=1 is a recipe for waiting far longer than you have to because of the overhead of the system calls. Better bs=N count=1
Re:Running Out of Disk Space in Production by __david__ on Tuesday April 07, 2026 @12:51PM (#47678146)
That’s also not great if you’re trying to make a 10 gigabyte file. In that case, use bs=1M and count=SizeInMB.
Re:Running Out of Disk Space in Production by marcosdumay on Tuesday April 07, 2026 @01:09PM (#47678387)
Modern computers are crazily overengineered...

Most current desktops (smaller than your usual server) won't have any problem with the GP's command. Yours is still better, of course.

Re:Running Out of Disk Space in Production by Twirrim on Tuesday April 07, 2026 @01:09PM (#47678381)
If you want to do it really quickly

    openssl enc -aes-256-ctr -pbkdf2 -pass pass:"$(date '+%s')" < /dev/zero | dd of=/home/myrandomfile bs=1M count=1024
Almost all CPUs have AES native instructions so you'll be able to produce pseudorandom junk really fast. Even my old system will produce it at about 3Gb/s. Much faster than urandom can go.
Re:Running Out of Disk Space in Production by ape4 on Tuesday April 07, 2026 @02:24PM (#47679328)
That's very cool. Sadly running that exact command gets an incomplete file and error "error writing output file". It suggests adding iflag=fullblock (to dd). Running that makes a file of the correct size. But still gives "error writing output file". I suppose that occurs because dd breaks the pipe.
Re:Running Out of Disk Space in Production by Twirrim on Tuesday April 07, 2026 @07:49PM (#47682820)
Weird, I could have sworn that used to work, maybe I wrote the notes down wrong.

Easiest alternative I guess is to pipe through head. It still grumbles, but it does work

    openssl enc -aes-256-ctr -pbkdf2 -pass pass:"$(date '+%s')" < /dev/zero | head -c 10M > foo
Re:Running Out of Disk Space in Production by tdeck on Tuesday April 07, 2026 @08:42PM (#47683233)
Fwiw you can also do this with

    head -c 1G /dev/urandom > /home/myrandomfile
And not have to remember dd's bizarre snowflake command syntax.
Re:Running Out of Disk Space in Production by bugfix on Tuesday April 07, 2026 @09:07PM (#47683465)
I just use fallocate to create a 1GB or 2GB file, depending on the total storage size. It has saved me twice now. I had a nasty issue with a docker container log quickly filling up the 1GB space before I could even identify the problem, causing the shell to break down and commands to fail. After that, I started creating a 2GB file.
Re:Running Out of Disk Space in Production by ape4 on Wednesday April 08, 2026 @09:12AM (#47689746)
Interesting! I did `fallocate -l 1G myfile` - very fast. Its all zeros but probably won't be compressed by the filesystem since its created with the fallocate() system call.
Re:Running Out of Disk Space in Production by dotancohen on Wednesday April 08, 2026 @01:36AM (#47685756)
My choice has always been `shred`:

  $ sudo truncate --size 1G /emergency-space
  $ sudo shred /emergency-space

I find it widely available, even in tiny distros.
Re:Running Out of Disk Space in Production by ape4 on Wednesday April 08, 2026 @01:56PM (#47693863)
Very cool. Reading the man page I see, by default, shred does 3 iterations. Adding --iterations=1 makes it random enough for this purpose and faster.
Re:Running Out of Disk Space in Production by dotancohen on Wednesday April 08, 2026 @07:56PM (#47697692)
Terrific, thank you!
Re:Running Out of Disk Space in Production by nyrikki on Tuesday April 07, 2026 @02:11PM (#47679154)
XFS, Ext4, btrfs etc… all support sparse files, so any app can cause problems you can try it with:

    dd if=/dev/zero of=sparse_file.img bs=1M count=0 seek=1024

If you add conv=sparse to the dd command with a smaller block size it will sparsify what you copy too, use the wrong cp command flags and they will explode.

Much harder problem than the file system layers to deal with because the stat size will look smaller usually.

Re:Running Out of Disk Space in Production by layer8 on Tuesday April 07, 2026 @03:20PM (#47680096)
Creating sparse files requires the application to purposefully use special calls like fallocate() or seek beyond EOF, like dd with conv=sparse does. You won't accidentally create a sparse file just by filling a file with zeros.
Re:Running Out of Disk Space in Production by nyrikki on Tuesday April 07, 2026 @04:08PM (#47680699)
It is an observability issue, even zabbix tracked reserve space and inodes 20 years ago.

Will dedupe,compression,sparse files you simply don’t track utilization by clients view, which is what du does.

The concrete implementation is what matters and what is, as this case demonstrates, is what you should alert on.

Inodes, blocks, extents etc.. are what matters, not the user view of data size.

Even with rrdtool you could set reasonable alerts, but the heuristics of someone exploding a sparse file with a non-sparse copy makes that harder.

Rsync ssh etc… will do that by default.

Re:Running Out of Disk Space in Production by zrm on Wednesday April 08, 2026 @04:34AM (#47687120)
A good way to do this is to create a swap file, both because then you can use it as a swap file until you need to delete it and because swap files are required to not be sparse.
Re:Running Out of Disk Space in Production by genewitch on Thursday April 09, 2026 @01:56AM (#47699792)
I'm not at a machine, linux doesn't zero the swap "file" does it? if i set vm.swappiness = 0, will your "trick" work if i never hit memory pressure?
Re:Running Out of Disk Space in Production by zrm on Thursday April 09, 2026 @02:19AM (#47699931)
If you create the file with 'mkswap --file' it allocates the blocks. Trying to use 'swapon' with an existing sparse file won't remove the holes for you but does notice them and then refuse to use it.
Re:Running Out of Disk Space in Production by PunchyHamster on Wednesday April 08, 2026 @06:55AM (#47688421)
No, just use LVM or other dynamic volume management.

Shit like that just wastes space that SSD could use for wear levelling...