News for hackers. Stuff that matters.
Re:Canonical, Ubuntu, and Why I Seem so Upset About Them All... by pekk on Thursday February 18, 2016 @11:00PM (#11131451)
If you have the source, it shouldn't be difficult to build binaries, should it?

If you are building a new Linux distribution based on Ubuntu, I don't even understand why you would want to reuse binaries they built.

This post would be more meaningful if it literally quoted from unfavorable license terms used by Ubuntu to actually prevent people from making derivatives. Saying that Canonical "appears to require" something is using weasel words.

Re:Canonical, Ubuntu, and Why I Seem so Upset About Them All... by mjg59 on Thursday February 18, 2016 @11:27PM (#11131533)
> If you have the source, it shouldn't be difficult to build binaries, should it?

You'd think, but no - there's no guarantee that the shipped binary packages can be built with the shipped toolchain or shipped dependencies.

> If you are building a new Linux distribution based on Ubuntu, I don't even understand why you would want to reuse binaries they built.

Because it's entirely unnecessary? Building the entire archive is a huge amount of effort.

> This post would be more meaningful if it literally quoted from unfavorable license terms used by Ubuntu to actually prevent people from making derivatives. Saying that Canonical "appears to require" something is using weasel words.

The terms are at http://www.ubuntu.com/legal/terms-and-policies/intellectual-... - I probably should have linked them, I've just been writing about this enough lately that it's easy to forget that people might read a single post without context.

Re:Canonical, Ubuntu, and Why I Seem so Upset About Them All... by ryao on Friday February 19, 2016 @12:35AM (#11131801)
> there's no guarantee that the shipped binary packages can be built with the shipped toolchain or shipped dependencies.

That problem will be with us as long as we use binaries as the primary means of distribution. If you consider that to be a problem, you could put effort into distributions where the packaging is the source code. I know I do.

> Because it's entirely unnecessary? Building the entire archive is a huge amount of effort.

How do you know that the source code provided actually corresponds to the binaries unless you compile them yourself[ with a toolchain you compiled yourself [...]]?

> The terms are at http://www.ubuntu.com/legal/terms-and-policies/intellectual-.... - I probably should have linked them, I've just been writing about this enough lately that it's easy to forget that people might read a single post without context.

No company's policies are perfect. You might be more effective trying to deal with the overall distribution of how software is done rather than imperfections in community leaders. There is a point of diminishing returns in focusing on a small subset of the companies out there while completely ignoring the rest.

Re:Canonical, Ubuntu, and Why I Seem so Upset About Them All... by mjg59 on Friday February 19, 2016 @01:27AM (#11131966)
> If you consider that to be a problem

I do consider it to be a problem, but since reality is problematic I think policies that force people to rebuild all binaries before they can redistribute them are harmful.

> How do you know that the source code provided actually corresponds to the binaries unless you compile them yourself[ with a toolchain you compiled yourself [...]]?

I don't, but I also don't have a verifiable path to bootstrapping a toolchain. I trust that Canonical haven't backdoored their packages.

> You might be more effective trying to deal with the overall distribution of how software is done rather than imperfections in community leaders.

I'm interested in it being straightforward for people to produce modified derivatives of the market leader, ie Ubuntu. I can achieve that in two ways:

1) Helping convince Canonical to change their IP policy 2) Helping convince Canonical to rearchitect their entire distribution infrastructure

(1) strikes me as being easier, so that's what I've chosen to work on.

Re:Canonical, Ubuntu, and Why I Seem so Upset About Them All... by pekk on Friday February 19, 2016 @01:17PM (#11135297)
If you are looking at some repository for Ubuntu's source code, and find that their build process isn't perfectly documented as regards the version of gcc they use, that really isn't a GPL violation no matter how hard you try to claim that it is.

You don't have a moral right to base a distribution on binaries produced by some other distribution. Unless you are going to produce actual evidence of a legal violation you really don't have anything here.

Re:Canonical, Ubuntu, and Why I Seem so Upset About Them All... by mjg59 on Sunday February 21, 2016 @02:07AM (#11143637)
Not sure whether this was aimed at me, but I don't believe that having source be difficult to build is a GPL violation.
Re:Canonical, Ubuntu, and Why I Seem so Upset About Them All... by ryao on Friday February 19, 2016 @01:55AM (#11132049)
> I don't, but I also don't have a verifiable path to bootstrapping a toolchain. I trust that Canonical haven't backdoored their packages.

You do not need a verifiable path to bootstrapping a toolchain to reduce surface area of attack to just the toolchain.

That being said, you could try building Clang with GCC and then GCC with Clang (or vice versa) to break any exploits designed to persist in either one indefinitely unless you are the victim of a really special hypothetical attack that targets both simultaneously, although I imagine you could interleave multiple GCC and Clang versions to make such an attack even harder to pull off or even add more compilers to the mix like EKOPath. That should reduce the attack surface area to binutils.

Re:Canonical, Ubuntu, and Why I Seem so Upset About Them All... by mjg59 on Friday February 19, 2016 @02:03AM (#11132078)
And this would still only buy me something if Canonical are backdooring their packages, which I don't think they are. It's a huge amount of extra effort for what is (as far as I'm concerned) zero benefit.
Re:Canonical, Ubuntu, and Why I Seem so Upset About Them All... by ryao on Friday February 19, 2016 @02:18AM (#11132116)
That assumes that anyone interested in putting backdoors into Canonical's packages never gets a job making their packages. If you are really concerned having trustworthy systems, you need to build from source and if you doing that anyway, you might as well just use the binaries built from source.
Re:Canonical, Ubuntu, and Why I Seem so Upset About Them All... by wantreprenr007 on Friday February 19, 2016 @01:08AM (#11131917)
Reproducibility from source and build env are fundamental to science or making changes to improve something. Without these, it can appear to signal hoarded proprietariness waving the banner of FLOSS or just a poorly-maintained project without high standards, even if it were unintentional.
Re:Canonical, Ubuntu, and Why I Seem so Upset About Them All... by ryao on Friday February 19, 2016 @01:57AM (#11132056)
> Reproducibility from source and build env are fundamental to science or making changes to improve something. Without these, it can appear to signal hoarded proprietariness waving the banner of FLOSS or just a poorly-maintained project without high standards, even if it were unintentional.

I am typing this on a system running Gentoo that has only 3 proprietary packages installed, with one being Intel's microcode. In time, I hope to lower that number to zero. If you are concerned about this and do not run a source-based distribution, I suggest that you switch.

Re:Canonical, Ubuntu, and Why I Seem so Upset About Them All... by majewsky on Friday February 19, 2016 @10:17AM (#11133882)
Source-based distributions are not the only solution. >> https://wiki.debian.org/ReproducibleBuilds
Re:Canonical, Ubuntu, and Why I Seem so Upset About Them All... by ryao on Friday February 19, 2016 @10:54AM (#11134189)
That requires making the builds easy to do, such that Matthew's claim that people cannot easily build things themselves probably is not valid. At least not if that project succeeds.

Anyway, I am happy to see that building from source is becoming easier for users of Debian and presumably Ubuntu by extension.

Re:Canonical, Ubuntu, and Why I Seem so Upset About Them All... by lmm on Friday February 19, 2016 @09:21AM (#11133558)
> You'd think, but no - there's no guarantee that the shipped binary packages can be built with the shipped toolchain or shipped dependencies.

If the shipped source doesn't contain enough information to actually perform a build surely that's a straight-up GPL violation (and a far more serious one than any possible ZFS issue, and one that you're in a position to do something about as a copyright holder). If the shipped source doesn't include whatever their developers actually use to build (and I refuse to believe an organization like Ubuntu wouldn't have a unified "build it all" script, or at least a README that described the steps you needed to take) then how can it possibly be the preferred form for making modifications?

Re:Canonical, Ubuntu, and Why I Seem so Upset About Them All... by mjg59 on Friday February 19, 2016 @12:14PM (#11134771)
> If the shipped source doesn't contain enough information to actually perform a build surely that's a straight-up GPL violation

Package A's source code contains some undefined behaviour that gcc 4.4 tolerates. The distribution upgrades to gcc 5.1 and the build now breaks, but nobody notices because no new version of Package A has been uploaded and so no new build has been performed. Is your position that it was in compliance with the GPL before the gcc transition, and in violation afterwards?

Re:Canonical, Ubuntu, and Why I Seem so Upset About Them All... by lmm on Friday February 19, 2016 @12:22PM (#11134837)
No. If the package A developers when working on the code just use whatever version of gcc's installed and don't have any process then they're compliant (but dumb). But if they have a script, or even just a "building.txt" that says "install gcc 4.4 and set these environment variables, build is broken on 5.x, known issue", and they exclude that from their source distribution then they're in violation. They need to ship the preferred form for making modifications i.e. what they would use themselves or give to a new developer on their team. It's no different from the example of C source generated by a perl script (you can't ship the generated source, you have to ship the original script), and it's exactly what's necessary for the four freedoms - end users need the same facilities that the original developers have when working on the code, not a watered-down version.
Re:Canonical, Ubuntu, and Why I Seem so Upset About Them All... by mjg59 on Friday February 19, 2016 @12:39PM (#11134998)
> But if they have a script, or even just a "building.txt" that says "install gcc 4.4 and set these environment variables, build is broken on 5.x, known issue", and they exclude that from their source distribution then they're in violation.

They don't.

Re:Canonical, Ubuntu, and Why I Seem so Upset About Them All... by lmm on Friday February 19, 2016 @12:49PM (#11135069)
Really? They being Canonical? They just build their binaries with whatever happens to be installed on Bob's machine this week and then that becomes their official release?
Re:Canonical, Ubuntu, and Why I Seem so Upset About Them All... by mjg59 on Friday February 19, 2016 @02:36PM (#11135974)
They build their packages with whatever is in the distribution at the time, yes. It happens on the build daemons rather than on the developer machine.
Re:Canonical, Ubuntu, and Why I Seem so Upset About Them All... by lmm on Friday February 19, 2016 @04:28PM (#11136763)
Then how can you get into the situation that the shipped binary packages can't be built with the shipped toolchain or shipped dependencies - isn't that what the build daemons do, use the system to build itself? I mean if they use the previous release or something that's fine too. Whichever way they do it, surely it's documented internally, because it's the kind of thing developers would need to know when developing. I can't believe it would be a case of "if the build doesn't build ssh into the build daemons and update them until it does".

I mean a developer working at Canonical, trying to make a change to one of the packages they're preparing for distribution, has to have some way to answer the question "what version of gcc is being used to build this package". Whether that information is kept in the source tree, on their wiki, or on a post-it on the employee fridge - in any case, it's part of the source in the preferred form for making modifications, because it's, well, part of what the employees use to make modifications.

Re:Canonical, Ubuntu, and Why I Seem so Upset About Them All... by mjg59 on Friday February 19, 2016 @06:38PM (#11137634)
> Then how can you get into the situation that the shipped binary packages can't be built with the shipped toolchain or shipped dependencies

The distribution isn't rebuilt every cycle. If no new source release has been uploaded, the existing binary will be used for the next release.

Re:Canonical, Ubuntu, and Why I Seem so Upset About Them All... by lmm on Sunday February 21, 2016 @03:45AM (#11143825)
So if they need to apply a small patch to a given package they'll sometimes discover that it was built 3 versions ago and no longer compiles? Yuck.
Re:Canonical, Ubuntu, and Why I Seem so Upset About Them All... by nkurz on Friday February 19, 2016 @05:51PM (#11137348)
Then how can you get into the situation that the shipped binary packages can't be built with the shipped toolchain or shipped dependencies

I think that Matthew might be suggesting that not all packages are rebuilt with every release? A binary package might be compiled with GCC-4, then the distribution is upgraded to GCC-5, dropping GCC-4. If for some reason the package is not compatible with the new compiler, you would then be in a situation where the binary cannot be recreated from source without outside tools. Do you know if Canonical recompiles everything on each update to the toolchain?

Re:Canonical, Ubuntu, and Why I Seem so Upset About Them All... by lmm on Monday February 22, 2016 @04:51AM (#11149708)
I have no specific knowledge about Canonical's processes. I'm just amazed that a serious software company in 2016 wouldn't have a reproducible build process for their primary product.
Re:Canonical, Ubuntu, and Why I Seem so Upset About Them All... by majewsky on Friday February 19, 2016 @10:15AM (#11133872)
There are scripts for building each package individually, and these scripts list other packages that are required for the build process. There is nothing to enforce the requirement that the build requirements are a subset of the packages built, or even correlated to them.
Re:Canonical, Ubuntu, and Why I Seem so Upset About Them All... by ghaff on Friday February 19, 2016 @10:00AM (#11133774)
>If the shipped source doesn't contain enough information to actually perform a build surely that's a straight-up GPL violation

Why would it be? Build and testing systems for something like a Linux distribution are far more complex systems than a simple script. In any case, there's no requirement in the GPL (or any other OSI-approved license AFAIK) to include components outside of the software itself to simplify building and installing. Of course, there are a variety of things like package managers that do simplify the process but there's no requirement in the GPL that it be straightforward to install from sources.

Re:Canonical, Ubuntu, and Why I Seem so Upset About Them All... by lmm on Friday February 19, 2016 @10:03AM (#11133787)
The requirement is to ship the source in the preferred form for making modifications. That doesn't include standardized external tools, but it surely includes the build instructions (with things like which versions of the tools to use) and test suite (and indeed git history - mjg59 has disagreed with me regarding that before, but I don't think he gave a rationale) - those are things you'd definitely want to have for making modifications.
Re:Canonical, Ubuntu, and Why I Seem so Upset About Them All... by vacri on Friday February 19, 2016 @05:41AM (#11132750)
mjg59: when you link to a Shuttleworth comment as a 'flat refusal', and that comment is actually a statement of position and procedure to achieve your general aims (if not your specific ones), but finally ends with a paragraph stating that you misrepresent what Shuttleworth says, it's hard not to believe him.
Re:Canonical, Ubuntu, and Why I Seem so Upset About Them All... by caf on Thursday February 18, 2016 @11:17PM (#11131507)
The post needs to be read in the context of the series of which it forms a part, eg. http://mjg59.dreamwidth.org/38467.html