News for hackers. Stuff that matters.
Re:Cmd/compile: Go 1.8 regression: sync/atomic loop elided by vessenes on Monday February 20, 2017 @09:36AM (#13687556)
This is interesting, actually.

As the language is specified, it's not wrong to just ignore that add completely -- the variable isn't used inside the goroutine, and you're not guaranteed a goroutine will run at any particular time.

So, code executed right after that routine is spawned will be hitting the 'old' value anyway; hence Dvyukov's comments that this is arguably to spec, or within spec.

Think if you actually wrote this code; you'd have some weird behavior if the compiler did what you 'expect' - a couple outputs of low integers then some high ones as the goroutine gets scheduled in. So it's almost certainly just 'wrong' to write something like this, under most (all?) circumstances.

Because sync.Wait() isn't used, there's no guarantee the goroutine will execute at all before program halt.

That said, I bet there's some related more reasonable scenario where you could argue the compiler should at least do the adding. I'm not sure why atomic.Add* should get special treatment though, and it looks like the fix is to remove some escape-analysis type code, so I'm curious how this will get fixed.

Re:Cmd/compile: Go 1.8 regression: sync/atomic loop elided by alecrn on Monday February 20, 2017 @09:53AM (#13687698)
Yeah, this particular instance seems ok to me. This one makes the example feel weirder:

  func main() {
  	runtime.GOMAXPROCS(runtime.NumCPU())
  	fmt.Println(runtime.NumCPU(), runtime.GOMAXPROCS(0))
  	started := make(chan bool)
  
  	go func() {
  		started <- true
  		for {
  			atomic.AddUint64(&a, uint64(1))
  		}
  	}()
  
  	<-started
  	for {
  		fmt.Println(atomic.LoadUint64(&a))
  		time.Sleep(time.Second)
  	}
  }
Here we explicitly wait until the goroutine is started, so we know it's scheduled by the time our other loop runs. Here on my computer with go1.8 linux/amd64 it still optimizes out the while loop, which makes sense as nothing changed that would convince the optimizer the loop should remain given the compiler's current logic.

If you add time.Sleep(time.Millisecond) to the goroutine loop or any other synchronization it works fine. I'm having trouble thinking of a real world example where you'd want an atomic operation going ham in a loop without any sort of time or synchronization. At the very least a channel indicating when the loop is done would cause the loop to compile.

Re:Cmd/compile: Go 1.8 regression: sync/atomic loop elided by schmichael on Monday February 20, 2017 @12:37PM (#13689165)
Just because started ensures the goroutine has been scheduled once it makes no guarantees it will ever be scheduled again. It really does nothing to extend the original example code.
Re:Cmd/compile: Go 1.8 regression: sync/atomic loop elided by thegeekpirate on Monday February 20, 2017 @07:07PM (#13692138)
FWIW, "runtime.GOMAXPROCS(runtime.NumCPU())" has been done automatically for you since Go 1.5, so there's no need to include it.
Re:Cmd/compile: Go 1.8 regression: sync/atomic loop elided by the8472 on Monday February 20, 2017 @10:02AM (#13687810)
> I'm having trouble thinking of a real world example where you'd want an atomic operation going ham in a loop without any sort of time or synchronization.

By using another atomic op as synchronization? After all that's their stated purpose.

Re:Cmd/compile: Go 1.8 regression: sync/atomic loop elided by alecrn on Monday February 20, 2017 @10:06AM (#13687844)
But I don't think in go atomic ops are considered synchronization in the sense that they force two goroutines to synchronize at a particular point like a chan. I.e. a chan send in one goroutine must be matched with a chan receive in another (unless they're buffered). If you have an atomic operation between two synchronization points I'd expect the only guarantee is that it occurs between the two points, and when it does it happens atomically.
Re:Cmd/compile: Go 1.8 regression: sync/atomic loop elided by flik on Monday February 20, 2017 @10:17AM (#13687925)
>>Strictly saying this is confirming behavior as we don't give any guarantees about scheduler (just like any other language, e.g. C++). This can be explained as "the goroutine is just not scheduled".

Not a GO developer. On a multi-processor machine, how is this a conforming behavior? Is "scheduler cannot give any guarantees" acceptable?

Re:Cmd/compile: Go 1.8 regression: sync/atomic loop elided by jerf on Monday February 20, 2017 @03:36PM (#13690669)
'Is "scheduler cannot give any guarantees" acceptable?'

Most schedulers give far fewer guarantees that you might think. A guarantee to be a guarantee must be true no matter what you do within the boundaries of the language. If you create a goroutine fork bomb

    func fork_bomb() {
        for {
            go fork_bomb()
        }
    }
Go doesn't, to the best of my knowledge, guarantee that any other goroutine will get any execution time, or guarantee much of anything will happen. Your OS is likely to have similarly weak guarantees for the equivalent process bomb, unless you do something to turn on more guarantees/protection.

You have to go into some relatively special-purpose stuff before you can get schedulers that will guarantee that some process will get scheduled for at least 10ms out of every 100ms or something. And then, once you get that guarantee, you'll pay some other way.

Given the fact that most of our machines are incredibly powerful, and that a lot of them still get upgraded on a fairly routine schedule in a lot of dimensions even if single-core clock speed has stalled out, most of us prefer to work with things that just promise to do their best as long as you don't overload them, because the other prices you have to pay to get guarantees turn out not to be beneficial on our monster machines. Of course one should always have their eyes out for when that may not be the case at some point, but in general we're headed away from rigorous guarantees in favor of shared resources that are cheaper and more scalable and making up the difference in volume, rather than trying to get better guarantees.

Re:Cmd/compile: Go 1.8 regression: sync/atomic loop elided by alecrn on Monday February 20, 2017 @10:24AM (#13687990)
My understanding is there are only scheduling guarantees around synchronization points between goroutines through chans or mutexes and stuff.

Go's concurrency model is inspired by Hoare's communicating sequential processes which kinda has the same idea: http://usingcsp.com/cspbook.pdf

For instance in this program:

  func main() {
  	point1 := make(chan bool)
  	point2 := make(chan bool)
  
  	go func() {
  		point1 <- true
  		fmt.Println("hello")
  		point2 <- true
  	}()
  
  	<-point1
  	time.Sleep(3 * time.Second)
  	<-point2
  }
An unbuffered channel read is always matched with a corresponding write. Let's call the point at which `point1 <- true` occurs and `<-point1` occurs T1 and the point at which `point2 <- true` occurs and `<-point2` occurs T2. fmt.Println("hello") and time.Sleep(3*time.Second) are both guaranteed to occur between T1 and T2. If we didn't have T2 there's no guarantee fmt.Println("hello") would run before the program exits.

Maybe I'm wrong but this is my understanding of Hoare's and go's concurrency model.

Re:Cmd/compile: Go 1.8 regression: sync/atomic loop elided by ridiculous_fish on Tuesday February 21, 2017 @02:40AM (#13694004)
How about a long-running calculation that uses atomic variables to report progress or poll for cancellation? Might it move all of the atomic ops outside of the loop?
Re:Cmd/compile: Go 1.8 regression: sync/atomic loop elided by Filligree on Monday February 20, 2017 @10:20AM (#13687949)
> Because sync.Wait() isn't used, there's no guarantee the goroutine will execute at all before program halt.

This is the sort of reasoning I'd expect to see from C apologists. Go should be better...

There are implicit guarantees on when goroutines are scheduled, too. The scheduler makes some attempts at fairly spreading the CPU.

Re:Cmd/compile: Go 1.8 regression: sync/atomic loop elided by knorker on Monday February 20, 2017 @12:28PM (#13689086)
> There are implicit guarantees on when goroutines are scheduled

But there's also explicitly no guarantees on that.

Re:Cmd/compile: Go 1.8 regression: sync/atomic loop elided by omginternets on Monday February 20, 2017 @07:14PM (#13692168)
This is unfair, I think. Go offers strong guarantees that C does not. It has fewer sharp edges than C; it also asserts a principle of not polishing away sharp edges with concurrency.

You don't have to do everything better than C to be a huge improvement over C.

Re:Cmd/compile: Go 1.8 regression: sync/atomic loop elided by fmstephe on Monday February 20, 2017 @09:45AM (#13687617)
Unfortunately this miscompilation probably is within spec, given that the atomic package has no documented ordering.

But it's coming.

https://github.com/golang/go/issues/5045

Re:Cmd/compile: Go 1.8 regression: sync/atomic loop elided by gpderetta on Monday February 20, 2017 @11:54AM (#13688754)
This is issue is not related to ordering, but to visibility (i.e., when is a memory side effect globally visible).

Many concurrent memory models (C++ at least, java very likely) allow coalescing stores to atomic variables as long as there is no other interleaving side effect; this is normally a desirable optimization.

As described by dvyukov, the go compiler seems to be aggressively coalescing all stores till the end of the loop (which never happens).

Store coalescing is desirable, but it is hard to specify completely, for example the C++ standard has non normative wording stating that stores should be visible in a reasonable amount of time exactly to prevent this sort of issues.

Re:Cmd/compile: Go 1.8 regression: sync/atomic loop elided by twic on Monday February 20, 2017 @04:03PM (#13690866)
Java only requires that the writes can't be seen out of order - if a thread writes A then B, another thread can't read the new value of B and then the old value of A. If coalescing them makes them all happen atomically, that's fine.
Re:Cmd/compile: Go 1.8 regression: sync/atomic loop elided by eternalban on Monday February 20, 2017 @09:53AM (#13687699)
What "'old' value" are you talking about?

This is a pointer:

    val := atomic.LoadUint64(&a)
> Because sync.Wait() isn't used, there's no guarantee the goroutine will execute at all before program halt.

It's an infinite loop ...

> I'm not sure why atomic.Add* should get special treatment though

Because it side effects and arguably it is a form (mem) I/O in context of modern systems.

Re:Cmd/compile: Go 1.8 regression: sync/atomic loop elided by 010a on Monday February 20, 2017 @10:28AM (#13688036)
> It's an infinite loop ...

In Go, if runtime.GOMAXPROCS() returns 1 or is set to 1 (meaning, Go only has access to a single system thread), the 'for' loop canabalizes the scheduler and the goroutine is rarely or never scheduled.

That fact doesn't make the OP's statement true, or have anything to do with the optimization bug, but it is worth pointing out that there are situations where Go correctly cannot guarantee that goroutine is executed.

Re:Cmd/compile: Go 1.8 regression: sync/atomic loop elided by eternalban on Monday February 20, 2017 @10:53AM (#13688241)
Yes, I am aware of that. Just addressing the OP's misread of the code.

But do note that /may/ change given the GC considerations:

https://github.com/golang/go/issues/10958

Re:Cmd/compile: Go 1.8 regression: sync/atomic loop elided by kevhito on Monday February 20, 2017 @05:40PM (#13691680)
I don't understand. By your argument, and some of the replies, because "you're not guaranteed a goroutine will run at any particular time", the compiler is free to elide any suffix of a goroutine's code, even if that code has visible side effects.

I think there is something more specific to atomic going on here, where the same kinds of optimizations are already forbidden for other kinds of visible side effects like printing.

Re:Cmd/compile: Go 1.8 regression: sync/atomic loop elided by omginternets on Monday February 20, 2017 @07:13PM (#13692165)
>even if that code has visible side effects

The argument is that you told it to do that. "Just print whenever", and whenever can mean never if the world dies prematurely.

Re:Cmd/compile: Go 1.8 regression: sync/atomic loop elided by kevhito on Monday February 20, 2017 @08:43PM (#13692549)
So you are telling me a valid optimization for Go is to elide the entire contents of all goroutines?

And by that argument, since main() itself is an implicitly-called goroutine (so far as I know), then the compiler can elide the entire contents of main() too, and then the rest of the contents of the program.

</reductio-ad-absurdum>

Re:Cmd/compile: Go 1.8 regression: sync/atomic loop elided by NateDad on Monday February 20, 2017 @11:12PM (#13693224)
Well, sort of, yes. https://play.golang.org/p/qyfqZA5pww

  func main() {
    go fmt.Println("Hi!")
  }
This is a valid go program (eliding import etc), and it'll almost never print anything.

This is actually a classic mistake when starting with goroutines - not adding synchronization to make sure your main goroutine doesn't exit before any spawned goroutines.

Re:Cmd/compile: Go 1.8 regression: sync/atomic loop elided by CasualSuperman on Saturday February 25, 2017 @09:54AM (#13731795)
Yes, but then the program would never terminate, because that's what happens at the end of main. If main returns, it's saying that the rest of the code in main has executed.