News for hackers. Stuff that matters.
Re:Don't setenv in multi-threaded code on glibc by quotemstr on Tuesday January 31, 2017 @05:23AM (#13529292)
Windows has sane environment-retrieval functions: https://msdn.microsoft.com/en-us/library/windows/desktop/ms6...

Unlike getenv, GetEnvironmentVariable copies a variable's value into a caller provided buffer, addressing the use-after-free problem inherent in getenv's interface.

Wouldn't it be nice if glibc got something like a getenv2 with a similar interface? Why should people have to trip over weird corner cases in POSIX over and over and over again? Why are we afraid to add new APIs that make some damn sense?

Re:Don't setenv in multi-threaded code on glibc by Sanddancer on Tuesday January 31, 2017 @05:32AM (#13529348)
This is a pure glibcism. FreeBSD libc does similar with getenv, in that the returned environment is a snapshot of that point in time.
Re:Don't setenv in multi-threaded code on glibc by simias on Tuesday January 31, 2017 @06:10AM (#13529584)
IMO getenv's API itself is to blame since it won't let you provide a buffer. As such if the libc wants to provide a local buffer it has to figure out a way to allocate it for the caller.

As far as I can tell glibc's implementation is probably what the people who designed getenv in the first place had in mind, just return a pointer to a global buffer. At best FreeBSD managed to create a workaround for modern environments.

There are unfortunately many such oddities in the standard libc APIs, remnants from an other time. Let us remember that "gets" was standardized as part of the C standard at some point, a function that's by design literally impossible to use safely.

Re:Don't setenv in multi-threaded code on glibc by cesarb on Tuesday January 31, 2017 @07:02AM (#13529822)
> Let us remember that "gets" was standardized as part of the C standard at some point, a function that's by design literally impossible to use safely.

It _is_ possible in theory to use gets safely, as long as your standard input is trusted. For instance, if a process forks and connects the standard input in the child to a pipe from the parent, which always writes a fixed amount of data to the pipe, you can use gets() without risk of overflow.

(This is a really contrived scenario, but I don't know of any simpler one where using gets() is safe.)

Re:Don't setenv in multi-threaded code on glibc by MaulingMonkey on Tuesday January 31, 2017 @12:47PM (#13533097)
> It _is_ possible in theory to use gets safely, as long as your standard input is trusted.

Only if we define "trusted" to include "known to be bug free" in e.g. it's truncation or bounding of output over the pipe to the child. I argue that, in theory, this is impossible to know, and thus that it this level of trust is impossible, and thus that this is not an example of a potential safe use of gets.

Even a mathematical proof of safety, after all, could contain errors - or could prove the wrong thing - or could apply to the code as written and not the code as messed with by your optimizer - or another thread - or an injected dll - or ...

> For instance, if a process forks and connects the standard input in the child to a pipe from the parent, which always writes a fixed amount of data to the pipe, you can use gets() without risk of overflow.

This is also insufficient - one must also prevent nonstandard invocations of the child process. Even if your normal parent process gives the child input that is 100% safe, that's no guarantee that an attacker won't launch your child process in an unusual manner. If the child process is suid, for example, this would be a potential avenue for privilege escalation.

Re:Don't setenv in multi-threaded code on glibc by cesarb on Tuesday January 31, 2017 @01:42PM (#13533753)
> Only if we define "trusted" to include "known to be bug free" in e.g. it's truncation or bounding of output over the pipe to the child. I argue that, in theory, this is impossible to know, and thus that it this level of trust is impossible, and thus that this is not an example of a potential safe use of gets.

"In the security engineering subspecialty of computer science, a trusted system is a system that is relied upon to a specified extent to enforce a specified security policy. As such, a trusted system is one whose failure may break a specified security policy." -- https://en.wikipedia.org/wiki/Trusted_system

That's the definition of "trusted" I'm using.

> This is also insufficient - one must also prevent nonstandard invocations of the child process.

I'm thinking of pure fork(), not fork+exec. That is, the child process is the same executable image, so the only way to invoke the child process in a nonstandard way would be through a debugger. And that is why the child process can trust the parent process in my example: they're the same process until the fork().

(As I said, it's a really contrived scenario. In more realistic scenarios, gets() is unsafe.)

Re:Don't setenv in multi-threaded code on glibc by comex on Wednesday February 01, 2017 @03:16AM (#13539220)
Well, on Linux, there are various hardening mechanisms that lock down ptrace, as well as things like /proc/PID/mem/, but I don't think the same restrictions apply to grabbing fds from other processes via /proc/PID/fd/. So in theory a process running as the same user (but which can't just debug your process due to hardening) could steal your pipe and exploit your program... probably not a terribly realistic threat, but a contrived objection to a contrived scenario :)
Re:Don't setenv in multi-threaded code on glibc by pjmlp on Tuesday January 31, 2017 @07:46AM (#13530038)
> Let us remember that "gets" was standardized as part of the C standard at some point, a function that's by design literally impossible to use safely.

Which thankfully was removed by recent standard revisions of C and C++, meaning a compliant compiler isn't required to provide it any longer.

Re:Don't setenv in multi-threaded code on glibc by JdeBP on Wednesday February 01, 2017 @06:20AM (#13539849)
Actually, what we should remember is that setenv, clearenv, unsetenv, and putenv came along a lot later than getenv. There was a fair while when the general mental model was that the environment was immutable.
Re:Don't setenv in multi-threaded code on glibc by exprA on Tuesday January 31, 2017 @08:26AM (#13530312)
gets is impossible to use safely with unformatted input, which need not be the case.
Re:Don't setenv in multi-threaded code on glibc by bonzini on Tuesday January 31, 2017 @06:53AM (#13529768)
That is irrelevant. FreeBSD getenv doesn't protect accesses to the event with a lock, meaning that getenv will access a freed copy of the environment if a concurrent call to putenv resizes the underlying array. Musl has the same issue.
Re:Don't setenv in multi-threaded code on glibc by Sanddancer on Tuesday January 31, 2017 @04:27PM (#13535412)
It uses a lock-free method of ensuring that there aren't segfaults. When it needs to move a variable because it's gotten too big, it copies it to the new location and sets the pointer accordingly with the old location flagged as old. The old variable isn't freed, so there are no dangling pointers, with the downside of memory usage being increased some.
Re:Don't setenv in multi-threaded code on glibc by bonzini on Tuesday January 31, 2017 @04:52PM (#13535586)
Doesn't it just use realloc? See __rebuild_environ in https://github.com/freebsd/freebsd/blob/master/lib/libc/stdl...
Re:Don't setenv in multi-threaded code on glibc by deathanatos on Tuesday January 31, 2017 @12:20PM (#13532747)
> FreeBSD libc does similar with getenv, in that the returned environment is a snapshot of that point in time.

How does FreeBSD's libc know when to free that snapshot? (Since there is no "free the env snapshot you gave me" call.) If thread B calls getenv, how does that not invalidate the snapshot that thread A has?

Re:Don't setenv in multi-threaded code on glibc by MaulingMonkey on Tuesday January 31, 2017 @01:02PM (#13533301)
Thread local storage would let you store a snapshot for thread A until thread A itself re-invokes getenv.

Other comments seem to imply this isn't what FreeBSD is doing however.

Re:Don't setenv in multi-threaded code on glibc by quotemstr on Tuesday January 31, 2017 @01:08PM (#13533361)
So what if you call getenv twice?

    const char* foo = getenv("FOO");
    const char* bar = getenv("BAR");
foo and bar should both be valid here. Sure, you could allocate an arbitrary amount of thread-local storage, but that's still a leak.
Re:Don't setenv in multi-threaded code on glibc by MaulingMonkey on Tuesday January 31, 2017 @01:13PM (#13533411)
Guess there's a reason FreeBSD isn't doing it :D.
Re:Don't setenv in multi-threaded code on glibc by pjmlp on Tuesday January 31, 2017 @06:13AM (#13529604)
Because as anyone doing POSIX development across multiple OS learns, that is an implementation detail.

POSIX like any other paper standard implies write once debug everywhere.

Re:Don't setenv in multi-threaded code on glibc by masklinn on Tuesday January 31, 2017 @09:35AM (#13530830)
> Because as anyone doing POSIX development across multiple OS learns, that is an implementation detail.

The problem is the interface is naturally unsafe. Much like e.g. strcpy, the interface means it's nigh impossible to use it safely.

Re:Don't setenv in multi-threaded code on glibc by pjmlp on Tuesday January 31, 2017 @01:04PM (#13533314)
Which is the reason that even if we would write an UNIX OS in something like Ada or Rust instead of C, we wouldn't get away from security exploits caused by the UNIX API.

Hence why in parallel with better systems languages, we really need tagged architectures for pointers, like what SPARC V8, Intel MPX, Cherry offer, to become widespread, for better security.

Re:Don't setenv in multi-threaded code on glibc by quotemstr on Tuesday January 31, 2017 @01:11PM (#13533392)
Upthread, I pointed out a sane and safe alternative to POSIX getenv interface, one that works fine with regular C, and here, you're claiming that we need exotic features to achieve safety? What?
Re:Don't setenv in multi-threaded code on glibc by pjmlp on Tuesday January 31, 2017 @03:08PM (#13534712)
Because you only focused on getenv.

I was replying about the whole UNIX API surface, regarding functions like strcpy.

You need exotic features to guarantee that the memory doesn't get corrupted.

Re:Don't setenv in multi-threaded code on glibc by tedunangst on Tuesday January 31, 2017 @10:36AM (#13531401)
The posix standard explicitly says setenv is not thread safe. This shouldn't be a mystery.
Re:Don't setenv in multi-threaded code on glibc by masklinn on Tuesday January 31, 2017 @01:45PM (#13533793)
Isn't the issue here that setenv breaks getenv rather than setenv "not being thread-safe"?
Re:Don't setenv in multi-threaded code on glibc by quotemstr on Tuesday January 31, 2017 @02:44PM (#13534478)
The issue is that getenv's API is fundamentally unsafe. setenv is fine.
Re:Don't setenv in multi-threaded code on glibc by quotemstr on Tuesday January 31, 2017 @01:12PM (#13533406)
But it should be. And we can make it thread safe very easily. Why the hell should we be limited by POSIX? Why can't we provide features beyond it?

It's access to the environ global that's inherently unsafe. setenv and a modified getenv can be accommodated just fine.

Re:Don't setenv in multi-threaded code on glibc by pjmlp on Tuesday January 31, 2017 @03:09PM (#13534720)
Portability across UNIX platforms.
Re:Don't setenv in multi-threaded code on glibc by quotemstr on Tuesday January 31, 2017 @03:36PM (#13534946)
So portability to you, means that the platform can't add features? I reject portability then.
Re:Don't setenv in multi-threaded code on glibc by pjmlp on Tuesday January 31, 2017 @05:05PM (#13535715)
To me? No, to everyone.

Lets say GNU/Linux does that, what you would do for other UNIX flavours, #ifdef soup?

The goal of standards is exactly to minimize that.

Re:Don't setenv in multi-threaded code on glibc by quotemstr on Tuesday January 31, 2017 @05:25PM (#13535901)
Ifdefs would be required. Or some kind of wrapper in gnulib. Or, like systemd, just non-portability. Your philosophy means stagnation forever. I reject that. Programs that use getenv today will not break. Objection to the existence of a getenv_s is absurd.

The only way we move forward is by trying new things, and we can't possibly get everyone to try a new thing all at once. If the thing ends up being useful, we standardize it.

Re:Don't setenv in multi-threaded code on glibc by pjmlp on Tuesday January 31, 2017 @12:56PM (#13533219)
Back when I cared about POSIX (early 2000's) the difference between what POSIX stated and what UNIX systems offered wasn't always the same.

I guess the situation might have improved.

Re:Don't setenv in multi-threaded code on glibc by JdeBP on Wednesday February 01, 2017 @06:24AM (#13539864)
Actually, it says that it need not be thread-safe. It does not prohibit a good implementation from making it thread-safe anyway.
Re:Don't setenv in multi-threaded code on glibc by JdeBP on Wednesday February 01, 2017 @06:44AM (#13539930)
> Why are we afraid to add new APIs that make some damn sense?

Speak for yourself. I implemented a Standard C library which guarded the environment, and various other global variables that the language requires that the library have, with mutexes. I did this in the early 1990s.

I am far from alone, as far as C library implementors go. Your "we" is actually a small number of people.

Re:Don't setenv in multi-threaded code on glibc by yuhong on Tuesday January 31, 2017 @02:16PM (#13534174)
This reminds me of GetEnvironmentStringsW, which used to return the pointer directly but was eventually changed (I think in Vista) to copy.
Re:Don't setenv in multi-threaded code on glibc by theamk on Tuesday January 31, 2017 @09:58AM (#13531020)
Agree. We have strerror_r -- why not add getenv_r?
Re:Don't setenv in multi-threaded code on glibc by quotemstr on Tuesday January 31, 2017 @01:09PM (#13533367)
I'd go write a getenv_r myself right now, but my gut feeling is that the glibc people would reject the patch on the basis that POSIX says that getenv is unsafe, we get along without a getenv_r, and therefore we don't need a getenv_r. This is why we can't have nice things.
Re:Don't setenv in multi-threaded code on glibc by alyandon on Tuesday January 31, 2017 @02:08PM (#13534068)
I don't know why you are getting downvotes. Someone tried to propose a getenv_r implementation for glibc in 2007 and even submitted a patch - which was rejected by a certain someone that is pretty famous for not wanting "bsd crap" in glibc.

https://sourceware.org/bugzilla/show_bug.cgi?id=4350

Re:Don't setenv in multi-threaded code on glibc by viraptor on Tuesday January 31, 2017 @08:57PM (#13537492)
Since he left, some old ideas have been raised again and merged. It could be worth revisiting.
Re:Don't setenv in multi-threaded code on glibc by alyandon on Tuesday January 31, 2017 @10:34PM (#13537996)
Ah, that's right he did move on. :)