News for hackers. Stuff that matters.
Re:Why would you still want to use strace in 2023? [video] by mlyle on Monday January 08, 2024 @02:44AM (#38909205)
strace is basically my first go-to when a command immediately fails with an obscure message.

strace, skim the last 4-5 pages of trace output before the error message is emitted, looking in particular for error return values (it's often not right before the error because the program has started to clean up). Figures out the problem 80% of the time within a couple of minutes.

Re:Why would you still want to use strace in 2023? [video] by charleshn on Monday January 08, 2024 @04:23AM (#38909781)
strace's -Z flag is quite useful to filter only syscalls that failed.

In a similar way, -k is useful to get tracebacks.

Re:Why would you still want to use strace in 2023? [video] by Two9A on Monday January 08, 2024 @04:56AM (#38910000)
I should write that down; -Z would've been really useful when my SSH server was falling over in a heap and I was left scrolling through strace to work out what happened.

I wrote that experience up last week, for what it's worth: https://imrannazar.com/articles/linux-upgrade-fail

Re:Why would you still want to use strace in 2023? [video] by iforgotpassword on Monday January 08, 2024 @02:46AM (#38909217)
And then IME, most of the time it's some file missing, or you can't figure out where it's getting its configuration from, so a grep for ^open is a good first try.
Re:Why would you still want to use strace in 2023? [video] by amelius on Monday January 08, 2024 @03:39AM (#38909509)
Isn't there a flag for getting only the opened filenames? (If not, why not)
Re:Why would you still want to use strace in 2023? [video] by zwp on Monday January 08, 2024 @03:48AM (#38909565)
You can use "-e trace=open" to trace only open(2) calls

Alternatively "-e trace=%file" to get all file-related system calls (will catch eg failing pre-emptive checks using access(3) -> stat(2)).

Re:Why would you still want to use strace in 2023? [video] by amelius on Monday January 08, 2024 @04:33AM (#38909824)
Ok, but then you will still need to parse the output to get the filenames. That's ok, but since it is something that is used a lot, you'd expect a flag.
Re:Why would you still want to use strace in 2023? [video] by tanelpoder on Monday January 08, 2024 @04:50AM (#38909959)
Check out strace -y and strace -yy (--decode-fds)
Re:Why would you still want to use strace in 2023? [video] by zwp on Monday January 08, 2024 @05:12AM (#38910100)
You still need to pull out the paths?

A sprinkling of grep/perl (awk/sed/ruby/...) is mostly good enough eg:

strace -e trace=%file cat /etc/passwd 2>&1 >/dev/null | grep ^open | grep -Po '(?<=").*(?=")'

Re:Why would you still want to use strace in 2023? [video] by eesmith on Monday January 08, 2024 @06:04AM (#38910437)
Is your example situation really all that common?

If so, what format do you expect for the output?

If it's one filename-per-line then how do you encode filenames with embedded newlines?

How do you encode non-UTF8 characters, or is the file meant to be parsed only in binary mode?

I don't know of any generally agreed upon spec for this, so no matter what you think is right, most people are going to have to write a special-purpose parser.

In which case you might as well parse the native strace output since one is about as complex as the other.

Re:Why would you still want to use strace in 2023? [video] by amelius on Monday January 08, 2024 @07:55AM (#38911271)
It can use the same format as the Unix find utility. This utility has a -print0 flag to separate filenames by NUL characters instead of newlines if desired.
Re:Why would you still want to use strace in 2023? [video] by eesmith on Monday January 08, 2024 @11:02AM (#38913906)
That is a good point.

I still don't see it as common use case.

Re:Why would you still want to use strace in 2023? [video] by c0l0 on Monday January 08, 2024 @06:25AM (#38910571)
FTR, on modern-ish glibc-powered systems (in code that actually does use libc, and does not do its very own syscall-related thing instead), you will not find a single call to open(2) issued, in my experience. That's because the library functions shadowing these syscalls were rewired to use openat(2) under the hood.

    $ strace -e trace=open cat /dev/null 
    +++ exited with 0 +++


    $ strace -e trace=openat cat /dev/null 
    openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
    openat(AT_FDCWD, "/usr/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
    openat(AT_FDCWD, "/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3
    openat(AT_FDCWD, "/dev/null", O_RDONLY) = 3
    +++ exited with 0 +++
Re:Why would you still want to use strace in 2023? [video] by lelanthran on Monday January 08, 2024 @09:27AM (#38912338)
If you want to catch both `open` and `openat`, the opensnoop BPF[1] program is pretty nifty, especially if you are trying to figure out file stuff across several different programs ("which #$%^-ing program keeps modifying this file", for example).

[1] I've been dipping my toes into BPF recently, and while complicated (best to simply clone the bpftools repo and work off of that) there's a lot that can be done that tools like strace won't be able to match.

Re:Why would you still want to use strace in 2023? [video] by ptsneves on Monday January 08, 2024 @04:35AM (#38909837)
Same here. I have a theory. The obscure error are due to unexpected Syscall results. Programmers are very aware of the states their programs can be into given the visible logic. They often are not aware of the possible states a syscall can be and their error modes. A pattern I see in python programs is the sys.exit and exception avoidance because they don’t want the user to see a stack trace. So they rely on conditionals and sys.exits, never considering and catching the exceptions that things like open and write can raise, leading ton exception traces anyway. This is one of the reasons strace sometimes is required. I have seen experienced programmers doing this pattern again and again and they are really convinced that it is helpful and consistent. This highlights the what I wrote above programmers often ignore the way external dependencies work, in this case the syscalls.

Another common syscall related issue strace helps is with permissions denied. Most scripts just defend against the file existing and not whether the operation is possible, leading to obscure messages like operation not permitted. Strace to the rescue in such cases

It is one oft first lines of debugging, even if I have the source code, as sometimes the syscall that fails is buried deep in library code, but with the failed syscall and arguments I can track the line that made the wrong call.

Re:Why would you still want to use strace in 2023? [video] by cwillu on Monday January 08, 2024 @05:28AM (#38910241)
> I have seen experienced programmers doing this pattern again and again and they are really convinced that it is helpful and consistent.

In a nutshell, this is why I hate working in go. While I'm sure your code, dear hn golang fan, doesn't suffer from this, much of the go code I've had to deal with extends the principle of “unaware of many common failure modes and swallow them into the exit code” to every function.

Re:Why would you still want to use strace in 2023? [video] by angra_mainyu on Monday January 08, 2024 @10:18AM (#38913144)
that's strange, I've been working professionally in Go for years and two things happen:

1) what are exceptions in many languages are plain errors in Go 2) a defer-recover block is usually put at the very top of a risky call

This covers 99% of sources of panics.

Now, the one that truly does mess you up is when it comes from CGO/calling out to C. That will tear your application and there is nothing you can do about it.

Re:Why would you still want to use strace in 2023? [video] by cwillu on Monday January 08, 2024 @11:11AM (#38914071)
I wish they paniced. I'm talking about trees of function calls with variations on “err = thing(); if err: return err” multiple times in a function, so receiving an error code in the root function tells you very little about what went wrong or where it went wrong.
Re:Why would you still want to use strace in 2023? [video] by gpderetta on Monday January 08, 2024 @03:09PM (#38917400)
Exactly! Throwing away the call stack before deciding how to handle the error is not ideal.
Re:Why would you still want to use strace in 2023? [video] by angra_mainyu on Thursday January 11, 2024 @01:34PM (#38956545)
I've honestly never needed it when dealing an error, the fact that errors are handled near where they occur means a stack is rarely needed.

If you bubble up errors a lot then errors.Is/As is made for that purpose.

You really only care about a stack trace with a panic.

In anycase, runtime.Stack() iirc should return the stack.

But this sounds more like trying to write other languages in Go rather than writing in Go.

Re:Why would you still want to use strace in 2023? [video] by sgarland on Monday January 08, 2024 @07:27AM (#38910992)
This is why for my Python programs, while I swallow exceptions when possible and turn them into user-friendly error messages that are likely to be the cause, I also add a -d,—-debug flag that ignores that and spits out everything. Because when you need to know, you need to know.
Re:Why would you still want to use strace in 2023? [video] by digitalsushi on Monday January 08, 2024 @07:34AM (#38911065)
People who need to be doing other things with their lives utter silent prayers of gratitude when they hope your code that does that exists, and does
Re:Why would you still want to use strace in 2023? [video] by lelanthran on Monday January 08, 2024 @09:08AM (#38912098)
> Most scripts just defend against the file existing

To me, any code calling a `DoesFileExist(filename)` type of function is a code smell.

There is no point to checking if the file exists. None. Ever![1]

There is never a case where this is useful. The only times I see code that checks for the existence of a file is when, on subsequent lines, the file is used as if it exists (no error-checking on open), which is a bug.

A line in code that checks for the existence of a file correlates, in my experience, very strongly with using that particular file in that specific buggy manner.

[1] You might say that it's to provide a better user experience, such as "Your config file does not exist, do you want to create it?" or similar, but that's still buggy if you're determining whether the file exists by checking existence.

Re:Why would you still want to use strace in 2023? [video] by mlyle on Monday January 08, 2024 @11:28AM (#38914373)
> There is no point to checking if the file exists. None. Ever![1]

> There is never a case where this is useful.

Only a Sith deals in absolutes. What about the use case when you change behavior when a sentinel file exists and don't care about the contents? ;) e.g. touch firstrun

It can also be useful if there's a whole bunch of prerequisites you're going to check before doing a bunch of work; yes, opening the descriptor and keeping it around might be nicer, but...

Also, what about if you want to not overwrite existing output if it exists? O_CREAT|O_EXCL could be better (and important to use if the overwrite check has security implications)... though that doesn't exist in fopen(3) and you'd need to use fdopen which is inconvenient and risky in its own ways.

Re:Why would you still want to use strace in 2023? [video] by lelanthran on Monday January 08, 2024 @01:02PM (#38915691)
> Only a Sith deals in absolutes.

Point taken :-)

> What about the use case when you change behavior when a sentinel file exists and don't care about the contents?

You'll still care that you can write that file, right?

> It can also be useful if there's a whole bunch of prerequisites you're going to check before doing a bunch of work; yes, opening the descriptor and keeping it around might be nicer, but...

But, after you've checked the prerequisites, and you start your main work, you're still going to have to handle the case in that main work that one of the files you checked has the wrong permissions/is locked/was deleted.

It's not unusual for users to accidentally open a program twice by misclicking it when they only opened it once.

> It can also be useful if there's a whole bunch of prerequisites you're going to check before doing a bunch of work; yes, opening the descriptor and keeping it around might be nicer, but...

You answered this question below for the `open`/`openat` syscall. For `fopen` ...

> though that doesn't exist in fopen(3)

It has been in the standard since C11, I believe - mode `"x"` for exclusive create and open. You can use it with most C implementations today as most systems support C11.

Re:Why would you still want to use strace in 2023? [video] by NekkoDroid on Monday January 08, 2024 @01:28PM (#38916023)
> You'll still care that you can write that file, right?

Sometimes you don't need it.

systemd itself just checks that `/etc/initrd-release` exists and runs in initrd mode changing its default target to boot into (IIRC you can also manually change the default target to `initrd.target` in the initrd, but this way the default systemd vendored files don't need to be touched).

https://github.com/systemd/systemd/blob/7f13af72f89452950226...

Re:Why would you still want to use strace in 2023? [video] by overbytecode on Monday January 08, 2024 @10:41AM (#38913543)
Okay..what is the alternative?
Re:Why would you still want to use strace in 2023? [video] by lelanthran on Monday January 08, 2024 @10:53AM (#38913755)
> Okay..what is the alternative?

Just ... open the file and handle the error?

What are you getting by checking that the file exists that you don't get from the 'file not found' error that the `openFile()` routine returns?

Because it doesn't matter if `doesFileExist()` returns true, you still have to handle the `file not found` error[1] when calling `openFile()` on the next line anyway.

[1] Just because the file exists when the program checked, that doesn't mean that the program can open it (permissions), that the file still exists (could have been removed between the call to check and the call to open), that the file is, in fact, a file that can be opened (and not a directory, for example), that the file is not exclusively locked (Windows) by some other process that opened it, etc. `doesFileExist()` tells you nothing that would change how the subsequent is written.

Re:Why would you still want to use strace in 2023? [video] by pixl97 on Monday January 08, 2024 @12:21PM (#38915146)
And if it's the initial configuration file?
Re:Why would you still want to use strace in 2023? [video] by lelanthran on Monday January 08, 2024 @12:54PM (#38915578)
> And if it's the initial configuration file?

You're still going to have to open it and handle the errors (in case it's the wrong permissions, or the filename is a directory, etc).

Checking if it exists doesn't make the subsequent code any easier or shorter - you still have to check for errors even if it exists.

Re:Why would you still want to use strace in 2023? [video] by overbytecode on Monday January 08, 2024 @01:39PM (#38916166)
That makes sense.

I guess we’re accustomed to thinking that absence of a file is not “exception-worthy” because it is expected under normal circumstances. But the cases you raised make sense.

Re:Why would you still want to use strace in 2023? [video] by hawski on Monday January 08, 2024 @04:46AM (#38909934)
I often use it to see what configuration files the thing reads, it is much more reliable than documentation.
Re:Why would you still want to use strace in 2023? [video] by bboozzoo on Monday January 08, 2024 @05:05AM (#38910054)
Since you almost always need to open a file in order to do anything useful `strace -vf -e openat` helped solve a lot of issues from simple configuration file lookup to ld.so library loading problems.
Re:Why would you still want to use strace in 2023? [video] by thequux on Monday January 08, 2024 @06:19AM (#38910537)
It is often useful to also watch for calls to the stat family of syscalls; some applications stat files that they wish to open rather than depending on the return code from open, so just looking for open will miss files that don't exist.
Re:Why would you still want to use strace in 2023? [video] by agumonkey on Monday January 08, 2024 @06:48AM (#38910706)
same, and I'm often surprised how many files are touched / read, and also how many times (trying to load local configs from dozens of places)..

some unix practices seems very inefficient

Re:Why would you still want to use strace in 2023? [video] by zh3 on Monday January 08, 2024 @10:37AM (#38913456)
strace solved an issue for me today - logging in to a different account on my debian PC using 'su -l guest' would hang for 30s before letting me in. Interestingly, 'strace su -l guest' failed with a permission error during the trace so looked like I was stuck ('sudo' didn't help).

Until I opened a 2nd terminal window and did 'while [ 1 ]; do sudo strace -p $(pidof su); done' - that looped until I tried the login again, then attached, followed the calls and showed the delay was due to elogind and dbus; a quick 'apt purge elogind' then fixed the slow login.