News for hackers. Stuff that matters.
Re:What's Coming in Go 1.15 by zaarn on Wednesday May 13, 2020 @03:19AM (#23164342)
The fun thing about the Go compatibility promise is that it doesn't work as well as the go developers probably imagine; just last week one of my older Go project was discovered to be broken; wouldn't compile anymore. golang.org/x/crypto was responsible, they used some code that wouldn't work in more modern go compilers anymore. Granted, I had not maintained the project in 3 years but I also had everything vendored just in case go decided to break anything upstream, and the broken parts didn't even touch on anything that the compatibility promise excludes.

I think the Go devs should improve in that section a bit.

Re:What's Coming in Go 1.15 by jeffsmith82 on Wednesday May 13, 2020 @03:50AM (#23164526)
I had the same issue. The issue was a security one https://nvd.nist.gov/vuln/detail/CVE-2019-11840 which required a breaking change to fix so their options where leave everyone vulnerable but could compile or break anyone that uses this code and fix it. I think they made the right choice.

Also golang.org/x/crypto is not a core library so doesn't actually fall under the compatibility guarantee but they seem to try their best not to break it.

Re:What's Coming in Go 1.15 by zaarn on Wednesday May 13, 2020 @04:02AM (#23164611)
Neither of those things applied though; I had x/crypto vendored, so it didn't change it's code either and the CVE is unrelated to the actual code that broke (I used blake2b) and again; it was vendored, so my code wouldn't break but be vulnerable.
Re:What's Coming in Go 1.15 by Scaevolus on Wednesday May 13, 2020 @04:06AM (#23164647)
How did vendored code that didn't change break your build?
Re:What's Coming in Go 1.15 by TheDong on Wednesday May 13, 2020 @04:43AM (#23164864)
The easiest way to accomplish that is definitely with //go:linkname

You can use a comment of the format '//go:linkname [localname] [method]' to link some function as another one. Notably, this includes unexported private methods in the go runtime and stdlib.

I've seen code in the wild that uses this to grab go's map hash algorithm, get monotonic time, and other things.

The go authors have used this hack themselves in the stdlib often enough because the runtime doesn't expose some knob, and rather than thinking that perhaps other developers may want that knob too, they instead use such hacks, but they also don't support said hacks (because how could you, obviously keeping all private functions stable forever is silly and impossible)

Re:What's Coming in Go 1.15 by zaarn on Wednesday May 13, 2020 @05:15AM (#23165015)
I updated the compiler.
Re:What's Coming in Go 1.15 by FiloSottile on Wednesday May 13, 2020 @11:31AM (#23167845)
That was golang.org/x/crypto's fault for using internals not covered by the compatibility promise [0], sorry about that.

However, we try to hold x/crypto itself to the same compatibility standard as the standard library, so I think simply updating x/crypto at the same time should have made it a smooth update.

[0]: https://news.ycombinator.com/item?id=23164795

Re:What's Coming in Go 1.15 by steeve on Wednesday May 13, 2020 @05:36AM (#23165121)
I just compiled a fairly complex (CGo etc...) project a few weeks ago which I hadn't compiled since 2014, and it just worked.

So I guess YMMV...

Re:What's Coming in Go 1.15 by zaarn on Wednesday May 13, 2020 @05:57AM (#23165238)
Of course, not every project will fail, but in my case it failed and I think that is the issue.
Re:What's Coming in Go 1.15 by steeve on Wednesday May 13, 2020 @06:03AM (#23165266)
Do you remember the specifics? I am not aware of syntax constructs that were removed in Go, I guess ever.

EDIT: I'm sad folks are downvoting for asking a question...

Re:What's Coming in Go 1.15 by zaarn on Wednesday May 13, 2020 @06:07AM (#23165286)
Older versions of the /x/crypto library no longer compile on newer go versions, there wasn't any indication this would happen and I vendored the library to avoid the API breakage in that library. From my standpoint, that completely ruins using that library as after 3 years either I vendored it and it breaks because of something the go compiler does or I didn't vendor it but the API changed. Either way, my program will break eventually and there is no way to avoid it.
Re:What's Coming in Go 1.15 by steeve on Wednesday May 13, 2020 @06:25AM (#23165407)
I'm sorry you had this experience. I'm genuinely curious as to what the breakage was so hopefully we can prevent it for some other folks, hence why I asked if you remembered the specifics.
Re:What's Coming in Go 1.15 by zaarn on Wednesday May 13, 2020 @09:01AM (#23166452)
Basically, x/crypto/blake2b received a patch to switch how it detects the presence of AVX and SSE at runtime, the symbols required for that to work with earlier versions where removed. While I understand this is technically covered in the compatibility promise, it's very frustrating when it happens with a library developed by google, especially when, at the time, the general recommendation was to vendor them so as to not suffer from breakage when they update.
Re:What's Coming in Go 1.15 by steeve on Wednesday May 13, 2020 @10:43AM (#23167338)
I believe this commit has background: https://github.com/golang/crypto/commit/d6449816ce06963d9d13...

To be fair though, that package was depending on a private API to begin with.

Re:What's Coming in Go 1.15 by coldtea on Wednesday May 13, 2020 @06:17AM (#23165359)
Well, that's for the better still.

If you use vulnerable crypto (willing to go with old unpatched vendored code) then you don't really need crypto at all..

Re:What's Coming in Go 1.15 by hibbelig on Wednesday May 13, 2020 @08:22AM (#23166109)
@zaarn explained in another comment that the vulnerability did not impact him, he did not exercise that code path.
Re:What's Coming in Go 1.15 by coldtea on Wednesday May 13, 2020 @02:01PM (#23169717)
I understand that, but it's not really relevant.

The code path that failed to build was still related to the included crypto (even if it was due to compiler change).

If that was their only way to fix the crypto issue, then it's totally fine, because that way they fixed it for people who did need and exercize the crypto code path -- and those take priority over people who "bundled some crypto code but didn't exercize that code path and their build still broke".

Re:What's Coming in Go 1.15 by zaarn on Thursday May 14, 2020 @07:34AM (#23178232)
I exercized a code path that broke, it's not like unrelated code broke, it was code I was actually using in a commit in the upstream library that changed not much for the end user and then was broken by the compiler fairly soon after.

The issue is related to AVX detection in the CPU, which has the reasonable failure path of "don't build it with AVX".

Re:What's Coming in Go 1.15 by scottlawson on Wednesday May 13, 2020 @04:25PM (#23171425)
Isn't /x/ experimental by definition? They introduced a breaking changed to an experimental library. This does not fall under the scope of the compatibility promise. The compatibility promise works great.
Re:What's Coming in Go 1.15 by todotask on Wednesday May 13, 2020 @03:33AM (#23164425)
One of those point under "Expectations" can be explain in this page

https://golang.org/doc/go1compat

Re:What's Coming in Go 1.15 by zaarn on Wednesday May 13, 2020 @03:49AM (#23164523)
Nope, none of these points apply to the specific change, atleast as far as I am able to tell. The code that broke did not use unsafe when I last updated it and then it simply didn't work anymore in the current go version.
Re:What's Coming in Go 1.15 by TheDong on Wednesday May 13, 2020 @04:45AM (#23164874)
I believe one of those probably applies.

> Use of package unsafe. Packages that import unsafe may depend on internal properties of the Go implementation. We reserve the right to make changes to the implementation that may break such programs.

I suspect the crypto code you vendored used unsafe somewhere within it (whether to use `//go:linkname` or for other reasons), which would except it from the go1 compatibility promise.

Re:What's Coming in Go 1.15 by todotask on Wednesday May 13, 2020 @03:58AM (#23164579)
I agreed with jeffsmith82's comment.
Re:What's Coming in Go 1.15 by zaarn on Wednesday May 13, 2020 @04:02AM (#23164619)
That comment isn't applicable either; I had x/crypto vendored so any fix upstream wouldn't break my app.
Re:What's Coming in Go 1.15 by Aissen on Wednesday May 13, 2020 @04:30AM (#23164795)
I'm afraid blake2 used assembly and runtime.support_avx2, which doesn't seem to be covered by the compatibility promise: assembly is part of the toolchain, and the symbol it used was an internal one.

See:

https://go-review.googlesource.com/c/crypto/+/110355/

https://go-review.googlesource.com/c/crypto/+/110016/

Re:What's Coming in Go 1.15 by zaarn on Wednesday May 13, 2020 @06:04AM (#23165271)
So, my code broke because a library that was provided by the go developers used some unstable feature I had no idea about and had no real way of knowing short of checking a dependency three levels below my project? And to top that, that library was certainly subject to API breakage, the entire reason I vendored it in the first place.

It doesn't seem reasonable to me. When I developed my project, vendoring /x/ projects was recommended exactly due to the breakage, so now it breaks my projects because of recommendations of the golang team.

Re:What's Coming in Go 1.15 by rat9988 on Wednesday May 13, 2020 @05:58AM (#23165242)
I'm pretty sure x packages are not part of go core.
Re:What's Coming in Go 1.15 by zaarn on Wednesday May 13, 2020 @06:05AM (#23165277)
x packages should still compile on newer go compilers in my opinion. Their compatibility isn't guaranteed if you don't vendor it, which was a recommendation when I started the project and as I pointed out in the comment you're responding to, it was vendored, so it didn't break itself.