News for hackers. Stuff that matters.
Re:Writing secure Go code by thinkharderdev on Monday November 04, 2024 @08:02PM (#42047696)
> ordinary code is memory-safe by default

What does that mean? What constitutes "ordinary"? I'm not sure there is any official definition of memory safety, but I would consider it to mean that aside from code that is explicitly marked as unsafe it is impossible to write code that has undefined behavior.

Re:Writing secure Go code by kccqzy on Monday November 04, 2024 @08:16PM (#42047756)
Good definition. I've seen Go beginners trying to append to a slice from multiple goroutines. It works as well as calling push_back on the same vector from multiple threads in C++. It can easily corrupt GC state and lead to segfaults. The beginner didn't use any advanced trickery or the unsafe package. Therefore Go is not a memory safe language.
Re:Writing secure Go code by sshine on Monday November 04, 2024 @08:33PM (#42047857)
> Therefore Go is not a memory safe language.

Interesting.

To quote the NSA [1], "Some examples of memory safe languages are Python, Java, C#, Go, Delphi/Object Pascal, Swift, Ruby, Rust, and Ada. Memory safe languages provide differing degrees of memory usage protections, so available code hardening defenses, such as compiler options, tool analysis, and operating system configurations, should be used for their protections as well."

The narrow definition of memory safety here is:

Go has garbage collection, so you won't have memory leaks or use-after-free.

Go is powerful enough that beginners can cause segfaults by accidentally abusing internals, okay.

I'm not sure this is a very redeeming property of Go: Being able to crash the GC, without the flexibility of manual memory management.

But I'm not sure I'd categorize it as "not memory safe" for the same reason C/C++ aren't (a trade-off).

Because I don't believe that you can generally leverage this for the kinds of memory exploits made in C/C++.

I recall that some ML dialects (Standard ML and OCaml) have a library function Obj.magic : 'a -> 'b which escapes the type system. Using this can easily cause segfaults. Does that mean Standard ML and OCaml are not memory safe? Generally, no, they're extremely safe if you avoid that feature, which is most likely. This is arguably less safe than Go, since you most likely won't accidentally run that function.

[1]: https://media.defense.gov/2022/Nov/10/2003112742/-1/-1/0/CSI...

Re:Writing secure Go code by kccqzy on Monday November 04, 2024 @08:42PM (#42047897)
I'm trying to provide some commentary to OP's original term of "ordinary code" three comments above. While this term is inherently ambiguous and subjective, my personal opinion is that appending to slices simultaneously from multiple goroutines count as "ordinary code" but Obj.magic does not.
Re:Writing secure Go code by tptacek on Monday November 04, 2024 @09:49PM (#42048177)
Yes. And: you will run into correctness bugs quickly if you mutate shared references in Go code. It's only my contention that you won't create a security vulnerability, in the colloquial understanding of the term (ie: a panic doesn't count).
Re:Writing secure Go code by lll-o-lll on Tuesday November 05, 2024 @12:58AM (#42048945)
If I can induce a race that corrupts a data structure so that it leaks data back to me that I shouldn’t have access to, does that count?
Re:Writing secure Go code by melodyogonna on Tuesday November 05, 2024 @02:01AM (#42049231)
How can you do that?
Re:Writing secure Go code by tsimionescu on Tuesday November 05, 2024 @02:47AM (#42049462)
You can, though it's much harder than in C or C++ or unsafe Rust for this to be exploitable. A data race on an interface value can give you a corrupted interface value, overwriting the vtable with struct contents. This can happen to lead to arbitrary code execution if you're unlucky enough, though in most cases it would be a SIGSEGV. It's also very hard for an attacker to craft a payload that can be guaranteed to reach this, though with a microservixe architecture with automatic restarts of failed services, they might get a lot of tries.
Re:Writing secure Go code by kaba0 on Tuesday November 05, 2024 @02:58AM (#42049532)
I mean, a very serious security vulnerability is/was row hammering, where an attacker was waiting on flipping a bit they have no access to by continuously flipping neighboring ones. Compared to that a race condition is "trivial" to exploit.
Re:Writing secure Go code by consteval on Tuesday November 05, 2024 @09:14AM (#42051599)
I don't think that's ordinary code. In any language, if you don't use a thread safe container and mutate it from multiple threads you'll get problems. This isn't an issue of memory safety but rather thread safety. You have to check the documentation for thread safe operations or otherwise use a lock. This goes for C#, Java, Go, you name it - the one singular exception being Rust. But, even Rust does not fully get around it.
Re:Writing secure Go code by kccqzy on Tuesday November 05, 2024 @01:27PM (#42053984)
You missed the point.

> In any language, if you don't use a thread safe container and mutate it from multiple threads you'll get problems.

Yes I agree there will be problems but what kind of problems do you get? Can you potentially get a memory safety problem, or are you guaranteed that the problem is not a memory safety problem?

The point is that thread safety problems in Go lead to memory safety problems. That's not the case in Java. You can crash the whole Go program by doing that, but you cannot crash the JVM by doing the same thing.

Re:Writing secure Go code by yawaramin on Tuesday November 05, 2024 @06:09PM (#42056001)
Crashing the whole program is actually memory safety. Because then the program can't get into an undefined state where parts of the program have access to memory they shouldn't.
Re:Writing secure Go code by kccqzy on Tuesday November 05, 2024 @09:25PM (#42056849)
Crashing via SIGSEGV is not memory safety.
Re:Writing secure Go code by consteval on Wednesday November 06, 2024 @10:47AM (#42064085)
Crashing with SIGSEGV can be perfectly memory safe. For example, writing to a NULL address is defined behavior on almost all platforms and typically causes the OS to send SIGSEGV.
Re:Writing secure Go code by kiitos on Wednesday November 06, 2024 @06:20PM (#42071101)
Appending to slices concurrently, without synchronization, is unambiguously invalid code.
Re:Writing secure Go code by capitol_ on Tuesday November 05, 2024 @03:02AM (#42049543)
Could you share more of your thoughts on why that kind of memory corruption wouldn't be exploitable? Do go have something in place that prevents it?
Re:Writing secure Go code by Cthulhu_ on Tuesday November 05, 2024 @04:05AM (#42049883)
To add to Go being memory safe, it automatically blanks/zeroes memory, unlike C.
Re:Writing secure Go code by everybodyknows on Monday November 04, 2024 @11:29PM (#42048610)
> corrupt GC state

I understand this to mean the runtime's internal state, not visible to user code. If so, in general we should expect almost any sort of crash mode to be possible. Seems fair enough to call this "memory-unsafe".

Re:Writing secure Go code by tptacek on Tuesday November 05, 2024 @02:27PM (#42054452)
You'll be using an idiosyncratic definition the rest of the industry does not use, but you do you.

What I think is happening here is another instance of a pattern that recurs all the time in communities like this: a term of art was created, "memory safety", to address the concept of languages that don't have buffer overflows, integer overflows, use-after-frees, double frees, controllable uninitialized pointers, and all the other memory lifecycle vulnerabilities. People unfamiliar with the state of the art heard the term, liked it, and have axiomatically derived their own definition for it. They like their definition better, and are not open to the idea that the term exists to serve a purpose orthogonal to their arguments.

Another recent instance of the same phenomenon: "zero trust".

Just as happened in the Zero Trust Wars of 2022, people, hearing the industry definition and intent of the term, scramble to reconcile their axiomatic definition with the state of the art, convincing themselves they were right all along.

The problem they have in this particular argument is: where are the vulnerabilities? Go is not a niche language. It is a high-profile target and has been for over a decade. I saw Go security talks at OWASP Chicago(!) in 2012(!). People have all sorts of hypotheses about how a memory corruption vulnerability --- not "memory corruption", but a vulnerability stemming from it, implying valuable attacker control over the result of whatever bad thing happened --- might sneak into a Go program. Practitioners hear those axiomatic arguments, try to reconcile them with empirical reality, and: it just doesn't hold up.

Just for whatever it's worth to hear this, if at Black Hat 2025 someone does to Go what James Kettle does to web frameworks ever year and introduces a widespread repeatable pattern of memory exploitability in Go race conditions, about half of my message board psyche will be really irritated (I'll have been wrong!), but the other half of my message board psyche will be fucking thrilled (there will be so much to talk about!) and all of my vulnerability researcher psyche will be doing somersaults (there will be so many new targets to hit!). On net, I'm rooting for myself being wrong. But if I had to bet: we're not going to see that talk, not at BH 2025, or 2026, or 2027. I'm probably not wrong about this.

Re:Writing secure Go code by thinkharderdev on Tuesday November 05, 2024 @03:10PM (#42054753)
> You'll be using an idiosyncratic definition the rest of the industry does not use, but you do you.

What definition are you using that you seem to think is the one definition of memory safety that is canonical?

> don't have buffer overflows, integer overflows, use-after-frees, double frees, controllable uninitialized pointers, and all the other memory lifecycle vulnerabilities

Any guarantees about this are dependent on the language not having undefined behavior in its safe subset. Once you have undefined behavior any other guarantees made about memory safety are significantly weakened.

> where are the vulnerabilities?

I don't know of any other than code written to demonstrate the concept. But I imagine if you look at any large Golang codebase you will find race condition bugs. So the fact that you have potential undefined behavior resulting from an extremely common coding error seems like it might be something to be concerned about (to me at least). Especially given how little Golang helps you write safe concurrent code.

That's not to say that Go is therefore totally useless and everyone should stop using it now because it's "insecure". But it also seems ... unwise ... to me to just pretend it's nothing because it is hard to exploit or that we don't have any (known) examples of it being exploited.

Re:Writing secure Go code by tptacek on Tuesday November 05, 2024 @03:16PM (#42054790)
You will find race condition bugs. You will not find memory corruption vulnerabilities. Go look.

The argument is not about whether languages admit vulnerabilities --- all of them do. The argument is about whether they admit the vulnerabilities that motivate the term of art "memory safety". Go does not, at least not in any non-contrived scenario not involving "unsafe" or FFI.

As for definitions, I like what Alex wrote about this; or, you can look at ISRG's writing about it.

https://alexgaynor.net/2023/oct/02/defining-the-memory-safet...

Re:Writing secure Go code by thinkharderdev on Tuesday November 05, 2024 @03:42PM (#42054978)
You linked me to an article that defines an unsafe language as:

"a programming language which, by default, allows code to introduce memory-related vulnerabilities (use after free, buffer over/under flow, use of uninitialized memory, type confusion) and undefined behavior,"

Which was my whole point. The "and undefined behavior" part is absolutely essential to the definition because you can't guarantee the first part without it.

> Go does not, at least not in any non-contrived scenario not involving "unsafe" or FFI.

It absolutely does. You can find examples of code that triggers undefined behavior in this thread. You can hand-wave them away as being "contrived" (which is literally correct) but that doesn't just make the problem go away.

We had a whole collective meltdown (pun intended) over spectre and meltdown a few years back even though AFAICT there have never been any known exploits in the wild. But most people who are knowledgeable in these things still took it seriously because they realized that:

1. Just because you haven't found a specific instance of an in-the-wild exploit doesn't mean there isn't one 2. You don't want to necessarily wait because if something it exploitable in theory is will almost certainly be exploited sooner or later. And it takes time to sort this stuff out

Is this relevant to your average working dev slinging micro-services in go? No, probably not and it's probably not something you should even think about for the most part. But if you're writing a complex, mission critical system with very hard security requirements? Yeah maybe you should worry about little bit about the fact that Go is, by the definition you yourself provided me, not a memory safe language.

Re:Writing secure Go code by tptacek on Tuesday November 05, 2024 @06:55PM (#42056274)
For a variety of reasons I'm confident in my interpretation of the link I provided.
Re:Writing secure Go code by fragmede on Tuesday November 05, 2024 @03:24PM (#42054845)
The problem with naming the language "go" is that my initial interpretation of your comment was to think that "go look" was a valgrind-esque tool for finding such things.

Too bad it's not.

Re:Writing secure Go code by Thaxll on Tuesday November 05, 2024 @12:28AM (#42048827)
Appending from multiple goroutine to an in un-synchronized slice is "memory safe", it's completely different from c/c++.

It behave exactly like Java or C# which are also memory safe.

Re:Writing secure Go code by tsimionescu on Tuesday November 05, 2024 @03:09AM (#42049574)
I'm not sure of C#, but Java has stronger memory guarantees than Go, even in the presence of a data race.

In Java, all primitive types (including Object pointers) are atomically modified. And since all Java writes are primitives (Java doesn't have structs), you can never corrupt a data structure at the Java level. Of course, you can still corrupt it at a logical level (break an invariant established in the constructor), but not at the language level.

Go has a guarantee that word-sized reads/writes are atomic, but Go has plenty of larger objects than that. In particular, interface values are "fat pointers" and exceed the word-size on all platforms, so interface writes are not atomic. Which means another thread can observe an interface value having a vtable from one object but data from another, and can then execute a method from one object on data from another object, potentially re-interpreting fields as values of other types.

Re:Writing secure Go code by tucnak on Tuesday November 05, 2024 @05:34AM (#42050348)
> Which means another thread can observe an interface value having a vtable from one object but data from another, and can then execute a method from one object on data from another object, potentially re-interpreting fields as values of other types.

If this were the case, then surely someone could construct a program with goroutines, loops and a handful of interface variables—that would predictably fail, right? I wouldn't know how to make one. Could you, or ChatGPT for that matter, make one for demo's sake?

Re:Writing secure Go code by kokada on Tuesday November 05, 2024 @06:21AM (#42050612)
I am also curious, I keep reading from this thread folks talking that this is possible, but I can't see to find anything searching in Google/DDG.

There is this document from Golang devs itself[1], that says:

> Reads of memory locations larger than a single machine word are encouraged but not required to meet the same semantics as word-sized memory locations, observing a single allowed write w. For performance reasons, implementations may instead treat larger operations as a set of individual machine-word-sized operations in an unspecified order. This means that races on multiword data structures can lead to inconsistent values not corresponding to a single write. When the values depend on the consistency of internal (pointer, length) or (pointer, type) pairs, as can be the case for interface values, maps, slices, and strings in most Go implementations, such races can in turn lead to arbitrary memory corruption.

Fair, this matches what everyone is saying in this thread. But I am still curious to see this in practice.

[1]: https://go.dev/ref/mem

Edit: I found this example from Dave Cheney: https://dave.cheney.net/2014/06/27/ice-cream-makers-and-data.... I am curious if I can replicate this in e.g.: Java.

Edit 2: I can definitely replicate the same bug in Scala, so it is not like Go is unique for the example in that blog post.

Re:Writing secure Go code by tsimionescu on Tuesday November 05, 2024 @02:03PM (#42054270)
> Edit 2: I can definitely replicate the same bug in Scala, so it is not like Go is unique for the example in that blog post.

Could you share some details on the program and the execution environment? Per my understanding of the Java memory model, a JVM should not experience this problem. Reads and writes to references (and to all 32 bit values) are explicitly guaranteed to be atomic, even if they are not declared volatile.

Re:Writing secure Go code by kokada on Thursday November 07, 2024 @03:30PM (#42080609)

    import java.util.concurrent.Executors
    import scala.concurrent.{ExecutionContext, ExecutionContextExecutor, Future}

    trait IceCreamMaker {
      def hello(): Unit
    }

    class Ben(name: String) extends IceCreamMaker {
      override def hello(): Unit = {
        println(s"Ben says, 'Hello my name is $name'")
      }
    }
    class Jerry(name: String) extends IceCreamMaker {
      override def hello(): Unit = {
        println(s"Jerry says, 'Hello my name is $name'")
      }
    }

    object Main {
      implicit val context: ExecutionContextExecutor = ExecutionContext.fromExecutor(Executors.newFixedThreadPool(2))

      def main(args: Array[String]): Unit = {
        val ben = new Ben("Ben")
        val jerry = new Ben("jerry")
        var maker: IceCreamMaker = ben
        def loop0: Future[Future[Future[Future[Any]]]] = {
          maker = ben
          Future { loop1 }
        }
        def loop1: Future[Future[Future[Any]]] = {
          maker = jerry
          Future { loop0 }
        }
        Future { loop0 }
        while (true) {
          maker.hello()
        }
      }
  }

Here. I am not saying that JVM shouldn't have a stronger memory model, after thinking for a while I think the issue is the program itself. But feel free to try to understand.
Re:Writing secure Go code by tsimionescu on Tuesday November 05, 2024 @09:50AM (#42051867)
Sure, here is an example:

https://go.dev/play/p/_EJ4EvYntr2

When you run this you will see that occasionally it prints something other than 11 or 100. If it doesn't happen in one run, run it again a few times.

An equivalent Java program will never print anything else.

Re:Writing secure Go code by tucnak on Wednesday November 06, 2024 @09:56AM (#42063259)
Thank you, that's really illuminating.
Re:Writing secure Go code by kaba0 on Tuesday November 05, 2024 @03:00AM (#42049539)
Not at all. Java or C# can end up in a logical bug from that, but they will never corrupt their runtime. So in java you can just try-catch whatever bad stuff happens there, and go on afterwards.

Go programs can literally segfault from a data race. That's no memory safety.

Re:Writing secure Go code by kccqzy on Tuesday November 05, 2024 @08:15AM (#42051242)
No. Go is worse than Java in this regard. I'll just give you a random example from my five seconds of search: https://github.com/golang/go/issues/37484
Re:Writing secure Go code by tptacek on Monday November 04, 2024 @08:14PM (#42047750)
An example of extraordinary code would be code that interfaces with and/or pulls in non-memory-safe legacy C code.

Another example would be code specifically contrived to highlight a soundness problem in the language.

I used the term "extraordinary" to avoid exactly this kind of bickering over corner cases that aren't relevant to day-to-day software development (or at least, not in ways that aren't immediately evident when they come up.)

Re:Writing secure Go code by thinkharderdev on Tuesday November 05, 2024 @06:23AM (#42050629)
> An example of extraordinary code would be code that interfaces with and/or pulls in non-memory-safe legacy C code.

That's my point though. Of course calling non-memory safe native code over FFI can lead to memory-safety problems in any language. Likewise using the "unsafe" subset that basically every language has. But none of that is required in Go. It is only required that you mutate shared state from different threads, which is something that I would imagine happens in a lot of Go code codebases since it is an extremely easy mistake to make.

To be clear I think:

1. Go is mostly a memory safe language because it does in fact prevent the most common memory safety issues in C/C++ (UAF, buffer overflows, etc)

2. It is LESS memory safe than other modern memory-sage languages (Rust, Java, C#, Python, etc....)

3. The memory safety issues in Go are very difficult to exploit in code that is not specifically crafted to surface them

Re:Writing secure Go code by danielheath on Tuesday November 05, 2024 @01:00AM (#42048952)
Go lets you use `unsafe.Pointer` (or indeed, assembly intrinsics) if you really want to, but those are certainly not used "ordinarily".
Re:Writing secure Go code by tsimionescu on Tuesday November 05, 2024 @03:11AM (#42049588)
It's not just about that. Data races can expose an object in a state that was never written from any thread in Go, potentially corrupting even internal details not exposed. Simply writing a struct value from two different threads can expose this.