News for hackers. Stuff that matters.

A comment on Linux /proc/pid/stat parsing bugs. Parent comment

Re:Linux /proc/pid/stat parsing bugs by xxpor on Thursday December 22, 2022 @12:41PM (#34096063)
Even if they insist on a file based interface (it is a UNIX, so fair enough), in modern times it would be nice if they used a "real" data format. Yeah, it's not like JSON parsers have never had bugs, but on average they'll be MUCH better than everyone and their mother hand rolling a C based bespoke parser. Obviously you'd need a new name to not break backwards compatibility.
[ Parent | on HN ]
Re:Linux /proc/pid/stat parsing bugs by kbrazil on Thursday December 22, 2022 @01:20PM (#34096573)
jc[0] supports proc files. Converts them to JSON or YAML. (I am the author)

[0] https://kellyjonbrazil.github.io/jc/docs/parsers/proc

[ Parent | on HN ]
Re:Linux /proc/pid/stat parsing bugs by henrydark on Thursday December 22, 2022 @02:01PM (#34097049)
I just had a quick read of the pid/stat parser, and the regex pattern starts with ^, but there's no $. Doesn't this mean that this parser suffers exactly the bug of the original post?
[ Parent | on HN ]
Re:Linux /proc/pid/stat parsing bugs by prattmic on Thursday December 22, 2022 @02:11PM (#34097200)
See https://news.ycombinator.com/item?id=34097179. A comm is limited to 64 bytes, so I don't think it is possible to fit a long enough comm to match the full regexp.
[ Parent | on HN ]
Re:Linux /proc/pid/stat parsing bugs by henrydark on Thursday December 22, 2022 @02:43PM (#34097593)
Right, it's not a security problem on its own, but it can make the regex not match at all causing jc to return an error. So jc suffers from the parsing bug mentioned in the post.

[edit:] In order to get jc to return an error one has to actually read the regex. Here is a file name that gets it to return an error:

  bad) S 1 2 3 4 5
[ Parent | on HN ]
Re:Linux /proc/pid/stat parsing bugs by kbrazil on Thursday December 22, 2022 @03:01PM (#34097871)
It doesn't look like jc suffers from this bug since the regex match is greedy:

    % echo '2001 (my (file) with) S 1888 2001 1888 34816 2001 4202496 428 0 0 0 0 0 0 0 20 0 1 0 75513 115900416 297 18446744073709551615 4194304 5100612 140737020052256 140737020050904 140096699233308 0 65536 4 65538 18446744072034584486 0 0 17 0 0 0 0 0 0 7200240 7236240 35389440 140737020057179 140737020057223 140737020057223 140737020059606 0' | jc --proc

    {"pid":2001,"comm":"my (file) with","state":"S","ppid":1888,"pgrp":2001,"session":1888,"tty_nr":34816,"tpg_id":2001,"flags":4202496,"minflt":428,"cminflt":0,"majflt":0,"cmajflt":0,"utime":0,"stime":0,"cutime":0,"cstime":0,"priority":20,"nice":0,"num_threads":1,"itrealvalue":0,"starttime":75513,"vsize":115900416,"rss":297,"rsslim":18446744073709551615,"startcode":4194304,"endcode":5100612,"startstack":140737020052256,"kstkeep":140737020050904,"kstkeip":140096699233308,"signal":0,"blocked":65536,"sigignore":4,"sigcatch":65538,"wchan":18446744072034584486,"nswap":0,"cnswap":0,"exit_signal":17,"processor":0,"rt_priority":0,"policy":0,"delayacct_blkio_ticks":0,"guest_time":0,"cguest_time":0,"start_data":7200240,"end_data":7236240,"start_brk":35389440,"arg_start":140737020057179,"arg_end":140737020057223,"env_start":140737020057223,"env_end":140737020059606,"exit_code":0,"state_pretty":"Sleeping in an interruptible wait"}
Edit: looks like I can tighten up the signature matching regex for the "magic" syntax per the issue found above. The greedy regex matching for the parser does seem to work fine, though.
[ Parent | on HN ]
Re:Linux /proc/pid/stat parsing bugs by kbrazil on Thursday December 22, 2022 @03:19PM (#34098092)
Interesting - the proc-pid-parser actually parses that file name just fine:

    $ echo '2001 (bad) S 1 2 3 4 5) S 1888 2001 1888 34816 2001 4202496 428 0 0 0 0 0 0 0 20 0 1 0 75513 115900416 297 18446744073709551615 4194304 5100612 140737020052256 140737020050904 140096699233308 0 65536 4 65538 18446744072034584486 0 0 17 0 0 0 0 0 0 7200240 7236240 35389440 140737020057179 140737020057223 140737020057223 140737020059606 0' | jc --proc-pid-stat

    {"pid":2001,"comm":"bad) S 1 2 3 4 5","state":"S","ppid":1888,"pgrp":2001,"session":1888,"tty_nr":34816,"tpg_id":2001,"flags":4202496,"minflt":428,"cminflt":0,"majflt":0,"cmajflt":0,"utime":0,"stime":0,"cutime":0,"cstime":0,"priority":20,"nice":0,"num_threads":1,"itrealvalue":0,"starttime":75513,"vsize":115900416,"rss":297,"rsslim":18446744073709551615,"startcode":4194304,"endcode":5100612,"startstack":140737020052256,"kstkeep":140737020050904,"kstkeip":140096699233308,"signal":0,"blocked":65536,"sigignore":4,"sigcatch":65538,"wchan":18446744072034584486,"nswap":0,"cnswap":0,"exit_signal":17,"processor":0,"rt_priority":0,"policy":0,"delayacct_blkio_ticks":0,"guest_time":0,"cguest_time":0,"start_data":7200240,"end_data":7236240,"start_brk":35389440,"arg_start":140737020057179,"arg_end":140737020057223,"env_start":140737020057223,"env_end":140737020059606,"exit_code":0,"state_pretty":"Sleeping in an interruptible wait"}
But the "magic" signature doesn't recognize it:

    $ echo '2001 (bad) S 1 2 3 4 5) S 1888 2001 1888 34816 2001 4202496 428 0 0 0 0 0 0 0 20 0 1 0 75513 115900416 297 18446744073709551615 4194304 5100612 140737020052256 140737020050904 140096699233308 0 65536 4 65538 18446744072034584486 0 0 17 0 0 0 0 0 0 7200240 7236240 35389440 140737020057179 140737020057223 140737020057223 140737020059606 0' | jc --proc             
    jc:  Error - Parser issue with proc:
                 ParseError: Proc file could not be identified.
                 ...
I can fix the "magic" signature (regex) to look for more parts of the stat file.
[ Parent | on HN ]
Re:Linux /proc/pid/stat parsing bugs by prattmic on Thursday December 22, 2022 @02:10PM (#34097179)
Neat! Your parser [1] almost has a similar issue because a comm could contain parenthesis, e.g., `foo) R 123 456`. But since a comm is limited to 64 bytes, I don't think it is possible to fit a fully matching string inside of the comm before the closing parent after the comm, which would thus make your regexp fail to match.

[1] https://github.com/kellyjonbrazil/jc/blob/master/jc/parsers/...

[ Parent | on HN ]
Re:Linux /proc/pid/stat parsing bugs by rdtsc on Thursday December 22, 2022 @03:23PM (#34098156)
> Yeah, it's not like JSON parsers have never had bugs, but on average they'll be MUCH better than everyone and their mother hand rolling a C based bespoke parser.

Currently, but if this idea started when Linux was become popular the real data format would have been XML. It might have been nice at the time, but today we would have laughed at it and said how outdated and silly it looks probably.

[ Parent | on HN ]
Re:Linux /proc/pid/stat parsing bugs by woodruffw on Thursday December 22, 2022 @03:35PM (#34098299)
People make fun of the excesses of the XML era, but that would still be an improvement in the state of affairs!

Realistically, however, a reasonable format here would be CSV/TSV with a bit of escaping, or even something like netstrings[1]. Either would be unambiguous, while retaining the shell-tool-friendly nature of these pseudofiles.

[1]: https://cr.yp.to/proto/netstrings.txt

[ Parent | on HN ]
Re:Linux /proc/pid/stat parsing bugs by fathyb on Thursday December 22, 2022 @06:10PM (#34100193)
This is pretty much what happened with Darwin from Apple. The kernel and even the bootloader uses .plist files [0] extensively, which are XML-based.

I like it. To change the PCI IDs for a driver for example, change the IOKitPersonalities property in the <Driver>.kext/Info.plist file.

[0]: https://en.wikipedia.org/wiki/Property_list

[ Parent | on HN ]
Re:Linux /proc/pid/stat parsing bugs by yuuta on Thursday December 22, 2022 @08:36PM (#34101339)
Indeed. Parsing files is a less robust way compared to calling some APIs or at least parsing some files with a schema (e.g. JSON or XML). For example, uptime(1) on Linux:

% strace uptime 2> /tmp/strace && grep proc /tmp/strace

17:35:24 up 3 days, 7:47, 1 user, load average: 2.29, 1.85, 1.56

openat(AT_FDCWD, "/usr/lib/libprocps.so.8", O_RDONLY|O_CLOEXEC) = 3

openat(AT_FDCWD, "/proc/self/auxv", O_RDONLY) = 3

openat(AT_FDCWD, "/proc/sys/kernel/osrelease", O_RDONLY) = 3

openat(AT_FDCWD, "/proc/self/auxv", O_RDONLY) = 3

openat(AT_FDCWD, "/proc/uptime", O_RDONLY) = 3

openat(AT_FDCWD, "/proc/loadavg", O_RDONLY) = 4

[ Parent | on HN ]
Re:Linux /proc/pid/stat parsing bugs by yuuta on Thursday December 22, 2022 @08:42PM (#34101375)
It may be better if we have something similar to (pseudocode) for kernel APIs:

  struct uptime_t u = {
  
      .time = 0
  
  };

  ioctl(open("/proc/uptime"), GET_UPTIME, &uptime);
[ Parent | on HN ]