News for hackers. Stuff that matters.
Re:GitHub responds to Dear GitHub letter by rogerbinns on Friday February 12, 2016 @10:32AM (#11087857)
I complain every 6 months about their "releases" page - eg this one for my project https://github.com/rogerbinns/apsw/releases

They unhelpfully add a "Source Code" link, which would be great if that is what it was. Instead it is actually just a zip file of the repository at that tag. But the repository is in a maintainer state, so the "Source Code" isn't useful to an end user because various tools need to be run (eg building help, automake/autoconf, dependencies). The people mislead by Github due to this "Source Code" contact me, not github.

Every six months for several years, I send the email explaining how this doesn't serve anyone's interest, how it hurts, and that I am happy with any solution (eg don't auto add, change the name to make it clear, be able to label an existing file as the "Source Code" etc). I get the usual response of understanding and sympathy, and vague suggestion that it may be addressed. Three years and counting ...

Re:GitHub responds to Dear GitHub letter by speg on Friday February 12, 2016 @12:04PM (#11088627)
> They unhelpfully add a "Source Code" link, which would be great if that is what it was. Instead it is actually just a zip file of the repository at that tag.

Huh, if all the files of the repository at that tag aren't the "source code" of that release what is?

Re:GitHub responds to Dear GitHub letter by maaku on Friday February 12, 2016 @12:38PM (#11088883)
Any reasonably complicated project has some pre-processing that has to be done for a source release. e.g. creating the configure script for autotools repos, or building the documentation. These steps can and should be automated, but github is not providing hooks to perform these automated tasks.
Re:GitHub responds to Dear GitHub letter by rattray on Friday February 12, 2016 @12:54PM (#11089001)
Wouldn't the output of that be the "distribution code" rather than the "source code"?

Do your projects not have a README telling people how to build? Are you targeting a nontechnical or beginner audience?

Re:GitHub responds to Dear GitHub letter by adrusi on Friday February 12, 2016 @01:10PM (#11089123)
There's a long history of distributing source code already partially processed, and for decades, this has been what people expected when they download a source tarball. Newer generations were introduced to open source through github and are less familiar with this, but they also are rarely interested in source tarballs — they're interested in either the source code to contribute to (in which case they'll git clone) or in a binary distribution.

Really the main purpose of source distributions is for packagers. Especially when you're creating pkgbuild scripts like for archlinux's AUR or gentoo or slackbuilds or macports or BSD, all the tools needed to build the source code become build-time dependencies for your package, which is annoying for people installing it.

Re:GitHub responds to Dear GitHub letter by snuxoll on Friday February 12, 2016 @01:54PM (#11089477)
Not having to add autotools to a BuildRequires is nice, but hardly the end of the world as a package maintainer. I agree though, the contents of the repository at a git tag is not necessarily a source DISTRIBUTION - which is what most people who are compiling your software would want - especially when many of them probably don't know the autotools incantations required to generate the configure script.
Re:GitHub responds to Dear GitHub letter by chei0aiV on Monday February 15, 2016 @06:24PM (#11106842)
We probably need to do away with partially processed source and distribute just the git repo (or a tarball without .git but with info about which git rev it is) and binary packages for end-users.
Re:GitHub responds to Dear GitHub letter by rattray on Friday February 12, 2016 @04:06PM (#11090588)
Gotcha, thanks for the explanation!
Re:GitHub responds to Dear GitHub letter by krisdol on Saturday February 13, 2016 @01:21PM (#11095010)
Shouldn't the person downloading the source code be expected to run those scripts? And topically those scripts are included in the source.
Re:GitHub responds to Dear GitHub letter by isavegas on Friday February 12, 2016 @05:34PM (#11091199)
Github isn't a build server. You shouldn't expect it to perform tasks for you. You should provide a CI server to your users if you want that functionality. I've never heard of downloading "source code" that has been preprocessed. The entire point of the repo is that it IS the source code. If you want it to be preprocessed for users, why even have the original form in the repo?
Re:GitHub responds to Dear GitHub letter by habitue on Friday February 12, 2016 @05:44PM (#11091254)
This is explained in reply to your sibling comment, but the convention on linux for many years is that the source distribution has automake/autotools already run. That way, when you download the source tarball, you know the process is `./configure && make` to make the final binary. The way package maintainers for distributions customize things is (hopefully) by providing flags for configure. The idea is that autotools should only be necessary for developers, not for maintainers.
Re:GitHub responds to Dear GitHub letter by HappyTypist on Friday February 12, 2016 @06:05PM (#11091344)
It is unfortunate, but just create a new branch and run autotools, and mark a release there.
Re:GitHub responds to Dear GitHub letter by vsl on Saturday February 13, 2016 @01:26AM (#11092866)
If by "unfortunate" you mean "totally crazy"... I'd rather deal with the occasional person confused by GitHub kids' ignorance of what source tarballs are meant to be that doing such wild and dirty dances in the repository.
Re:GitHub responds to Dear GitHub letter by oneeyedpigeon on Saturday February 13, 2016 @07:09AM (#11093662)
TL;DR source code to install from and source code to hack on are often different beasts.
Re:GitHub responds to Dear GitHub letter by IshKebab on Friday February 12, 2016 @12:57PM (#11089024)
Yeah but those steps should be part of the normal build process. If you have to use weird tools (e.g. autotools) to generate your "real" source code, I'd say the problem is those tools. Also. Seriously. Autotools? In 2016?
Re:GitHub responds to Dear GitHub letter by scrollaway on Friday February 12, 2016 @01:50PM (#11089454)
Who's going to volunteer porting, say, Wine's 4000 line autotool script[1] to CMake, exactly? You, maybe? (I'm not being sarcastic, if you actually came up with a patch it might be considered...)

[1] http://source.winehq.org/git/wine.git/blob/0f8a0fd4002f9d5d1...

Re:GitHub responds to Dear GitHub letter by CydeWeys on Friday February 12, 2016 @01:12PM (#11089148)
Autotools? In my Linux?

It's more likely than you think.

Re:GitHub responds to Dear GitHub letter by adrusi on Friday February 12, 2016 @01:52PM (#11089464)
So what would you suggest in lieu of autotools? The only real mature alternative is cmake, and cmake's main advantage is that it is a cross platform build system. Basically that means it works on windows without cygwin or its equivalents.

If you use something other than cmake or autotools, you're going to end up making the lives of packagers more difficult. If they have to change anything, as they usually do, they're going to have to know how the build system works, and they dont necessarily want to deal with whatever new-fangled build system you're using.

Usually projects using cmake distribute their source code with cmake build scripts because they're taking advantage of cmake's cross plaform features, and cmake generates different source distributions for different platforms. But this does place a burden on the user to have cmake installed. There's a good argument to be made that that's no longer much of a burden, after all, it's just one package manager invocation away. But you have to think about who's downloading a source distribution (as opposed to binaries or checking out the repository). They're people who are compiling from source for esoteric platforms, perhaps and embedded system, where a cmake package might not already exist. They're people like me who are writing their own pkgbuild script because they like to keep their system well organized and not have too many useless packages (like cmake) lying around. They are people creating a new linux distribution who haven't yet packaged cmake but want to package your software.

And in terms of software design, cmake isn't very good, at least in the eyes of many people writing unix systems software, which as far as mature open source software projects written in C/C++ go, is a lot. Don't get me wrong, cmake certainly has the cross platform advantage, and if you're making software that you want to compile on windows, you'd be remiss to not at least seriously look into using cmake (not that it's the only option, you could also opt to lean on cross-compiliation from a unix platform to windows, distributing binaries only, or requiring users to install cygwin, which isn't worse than having to install visual studio if you don't already use it).

But if you're not taking advantage of the cross-platform features, I'd say cmake is pretty awful. Precisely because it's cross-platform, cmake rejects the typical design principles of unix software, instead offering a monolithic build system that requires special scripts to be written for any different tools you want to use, or even any complicated dependencies (for example: SDL). This is necessary because cmake needs to establish a consistent internal interface that can work for both the windows and unix tools. As a result, everyone has to learn how to use the monolithic cmake system from the ground up, and those skills won't be useful for anything else.

In contrast, autotools, known for having a high learning curve, is really not that bad for people who already know unix. If you understand how to use a unix system, you can figure out autotools faster than you can cmake. And you can really know it, to the extent that you feel confident diving in and messing with its defaults, much sooner than you would with cmake.

But of course, ease of learning isn't the only factor autotools integrates with external tools using their standard interface while cmake demands you essentially write a wrapper for them. Autotools is simply more flexible.

I don't like autotools, for the record, but given the options available, I'd probably choose it over any other build system in almost all the places it's used currently and more.

Re:GitHub responds to Dear GitHub letter by GFK_of_xmaspast on Friday February 12, 2016 @04:44PM (#11090886)
> In contrast, autotools, known for having a high learning curve, is really not that bad for people who already know unix. If you understand how to use a unix system, you can figure out autotools faster than you can cmake. And you can really know it, to the extent that you feel confident diving in and messing with its defaults, much sooner than you would with cmake.

I've been using unix since 1992 and find the entire autotools suite to be almost incomprehensible. The low point was in 2014 when I was trying to compile a package that intentionally did not ship a configure file and made the user explicitly call autoconf. This would have been ok except that in the 3 or so years between when the code was released and I was trying to build it, autotools made some backward-incompatible changes and autoconf/automake errored out, with no indication of how to fix those. Eventually I just gave up and used a different package.

Conversely, when I started using cmake (motivated by clion using it as a build system) I was able to get it up and running basically immediately, and 12 months later I appear to have become the cmake guru for my company.

Re:GitHub responds to Dear GitHub letter by mcguire on Friday February 12, 2016 @04:59PM (#11090996)
"The low point was in 2014 when I was trying to compile a package that intentionally did not ship a configure file and made the user explicitly call autoconf."

At the risk of having to admit I've had my share of bad experiences with autotools, don't ever do that. That is a big sign with flashing lights and sirens, saying that you should put that package down and leave it alone. Because...

"This would have been ok except that in the 3 or so years between when the code was released and I was trying to build it, autotools made some backward-incompatible changes and autoconf/automake errored out, with no indication of how to fix those."

The version of autotools someone was using when they wrote the scripts is very closely tied to the scripts. It's not at all a good idea to use a different version of the autotools.

Re:GitHub responds to Dear GitHub letter by cdcarter on Saturday February 13, 2016 @12:11AM (#11092658)
> At the risk of having to admit I've had my share of bad experiences with autotools, don't ever do that. That is a big sign with flashing lights and sirens, saying that you should put that package down and leave it alone. Because...

And indeed, the exact issue the OP is complaining about is how the zipfile automatically created by GitHub has NOT had autotools run on it yet, and the maintainer cannot even manually overwrite that file!

Re:GitHub responds to Dear GitHub letter by csense on Friday February 12, 2016 @04:57PM (#11090980)
I have to respectfully disagree. Cmake isn't super amazing and has some flaws and a less than ideal design and language, but it gets the job done. Autotools is more like something that was needed once in the 1980's when you had to support 300 different proprietary UNIX's and compilers / stdlib's weren't very standardized and had lots of bugs and warts, and now only exist because of inertia.
Re:GitHub responds to Dear GitHub letter by vsl on Saturday February 13, 2016 @01:29AM (#11092877)
I too have to disagree. Cmake is crap compared to Autotools. It's makefiles don't work if the end user who does the compilation doesn't have cmake installed. Autotools doesn't have any runtime dependencies - that's a design goal that yes, still matters 2016.
Re:GitHub responds to Dear GitHub letter by david-given on Friday February 12, 2016 @12:23PM (#11088756)
Source distribution packages aren't necessarily a simple snapshot of the repository.

For example: for autotools based build systems you want to run autoconf and automake to generate the Makefile.in and configure scripts.

Re:GitHub responds to Dear GitHub letter by striking on Friday February 12, 2016 @12:14PM (#11088690)
A ZIP file is less helpful than the GitHub interface pointed at that specific commit ID.
Re:GitHub responds to Dear GitHub letter by JonathonW on Friday February 12, 2016 @01:30PM (#11089291)
It's not immediately obvious, but the tag and commit links to the left of the release (under the green "latest release" label) actually do that.

The tag link points to the tree (in Github's file browser UI) for the release, and the commit ID link points to the specific commit for that tag (in Github's diff UI).

Trouble is, this isn't very discoverable, given that those two links aren't styled as links-- they're the same font and color as is used for the subheading with date and commits since, which aren't clickable elements.

Re:GitHub responds to Dear GitHub letter by striking on Friday February 12, 2016 @01:35PM (#11089337)
Sure. The issue is less with the functionality and more with how it's presented to the user.
Re:GitHub responds to Dear GitHub letter by Sir_Cmpwn on Friday February 12, 2016 @10:53AM (#11088044)
I also have a problem with the "download zip" button on repositories. It doesn't include submodules. I've asked Github several times to give me a button to remove it from my repo, or to add submodules, to no avail.
Re:GitHub responds to Dear GitHub letter by rogerbinns on Friday February 12, 2016 @11:04AM (#11088148)
I'll bet they have zero positive feedback over the Source Code zip, and you confirm yet more negative feedback over it.
Re:GitHub responds to Dear GitHub letter by morgante on Friday February 12, 2016 @11:53AM (#11088541)
I like having it as a simple way to quickly download a git-free tree of simple projects. So there's one small piece of positive feedback.
Re:GitHub responds to Dear GitHub letter by rogerbinns on Friday February 12, 2016 @10:47PM (#11092411)
You can already get such an archive by clicking on the right bit of the release page. And I believe at any page reflecting a revision. It is easy to get a copy of the repository.

It isn't the case for all projects, but for some there are some pre-build steps that the majority of people who want to install and use the project don't have or even know about. The project can put up the source after those steps are done, and the downloader does the final source building and installation with ease.

As an end user looking at the release page, you can't tell the difference between the easy way (the pre-processed source that will work well for you) and the hard way (you'll need to install and invoke several tools for no real benefit).

The underlying problem is that Github make it really easy for people to end up with the hard way, and not the intended easy way. This benefits no-one. I have no problem with them making the repository files available - I just want them to stop confusing people who go to a release page.

Re:GitHub responds to Dear GitHub letter by morgante on Saturday February 13, 2016 @05:16AM (#11093436)
I'm not talking about the releases page, and I don't think @Sir_Cmpwn was either.

Many small projects don't have a releases page. The download zip is just a way of quickly getting the source code for such projects.

To be honest, I wasn't trying to start an argument. I was just pointing out that I like having a "download zip" button.

Re:GitHub responds to Dear GitHub letter by rogerbinns on Saturday February 13, 2016 @10:35AM (#11094303)
Understood. I wasn't clear that I have no problem with being able to download a copy of the repository from as many pages/places as is reasonable, and there is no need to change that.

The specific top level complaint is about the releases page when visited by a human who would have the goal of building and installing the software release. There the "Source Code" link is misleading if the maintainer has a release file that is the expected one for building and installing the software.

Re:GitHub responds to Dear GitHub letter by kevin_thibedeau on Friday February 12, 2016 @07:53PM (#11091823)
It's also immensely useful for installers like pip that can pull an archive directly from Github using just a URL and no local git install.
Re:GitHub responds to Dear GitHub letter by rogerbinns on Friday February 12, 2016 @10:50PM (#11092417)
This issue is not about tools, but about humans being confused by the releases page. I have no problem with an archive of the repository being available - the issue is that github ui on the release page implies that is the best download to build and install the software, when it can be the least convenient hardest way.
Re:GitHub responds to Dear GitHub letter by recursive on Friday February 12, 2016 @12:29PM (#11088804)
Personally, I'm a big fan of the "Source Code" zip file download. I use it regularly, and have never had a problem.
Re:GitHub responds to Dear GitHub letter by rogerbinns on Friday February 12, 2016 @10:53PM (#11092430)
There is no problem with a way of grabbing the repository contents. Good to have, and shouldn't go away. It is the github ui on the release page strongly suggesting to use that download, even when the project has provided a better archive for users to build and install the software. That better archive can already have several pre-build steps run so the end user doesn't need additional tools of no benefit to them.
Re:GitHub responds to Dear GitHub letter by recursive on Monday February 15, 2016 @09:42PM (#11107586)
To be honest, I didn't even know there was a release page in github repos. I don't recall seeing those. I normally don't expect to be able to download installers from github, so I never really thought to look for such a thing.
Re:GitHub responds to Dear GitHub letter by aardshark on Friday February 12, 2016 @02:02PM (#11089557)
I use it semi-frequently and have never had a problem.
Re:GitHub responds to Dear GitHub letter by wnevets on Friday February 12, 2016 @10:35AM (#11087879)
>They unhelpfully add a "Source Code" link, which would be great if that is what it was. Instead it is actually just a zip file of the repository at that tag.

I fall for this everytime I go to use that link.

Re:GitHub responds to Dear GitHub letter by danielweber on Friday February 12, 2016 @11:15AM (#11088250)
The people who dogfood Github, by using it every day, know how to get around places.

The people who is it once every week or two, like me, get lost each time. It's always a hunt when I want to figure out how to do something, even if I know I've done it before.

Re:GitHub responds to Dear GitHub letter by haberman on Friday February 12, 2016 @12:14PM (#11088688)
Totally agree.

A few examples for me:

- From a "pull request / files" view like this (https://github.com/google/protobuf/pull/1191/files), when I want to view a whole file, I always want to click on the filename. But the filename isn't a link: you have to look at the right side and click the button that is generically named "View" (View what?).

- At the top level of a repo (ie https://github.com/google/protobuf), you can click "X commits" to view the list of commits. But when you go into a subdirectory (ie. https://github.com/google/protobuf/tree/master/benchmarks) the "commits" link disappears, and to view the commits in that directory you have to look on the right side instead for a button that says "History".

- continuing from my last point, if you view an individual file, (ie. https://github.com/google/protobuf/blob/master/benchmarks/Pr...), the "History" link is in yet another place! So between the top level, a subdirectory, and a file, the link to view history is in three different places. I don't follow these links quite often enough that it becomes automatic, so it slows me down every time.

Re:GitHub responds to Dear GitHub letter by ValentineC on Friday February 12, 2016 @01:06PM (#11089089)
I'm wondering what's a good way for a dogfooding company to look at things from a fresh perspective. Perhaps the traditional focus group and user interaction studies?
Re:GitHub responds to Dear GitHub letter by hussong on Friday February 12, 2016 @03:44PM (#11090397)
Listen closely to anything new team members have to say. From the inside, it all makes sense, but from the outside, it's often obvious insanity ;-)
Re:GitHub responds to Dear GitHub letter by radarsat1 on Friday February 12, 2016 @03:48PM (#11090438)
Glad someone else acknowledges this.. the number of times I've had to answer questions about how to get automake to work, etc., from people who should be just downloading the tar and running `./configure`. People are starting to get this tendency to check out the whole repository just to use the software. (I've been guilty of this too! There is something about it..) But people: If you're not going to work on the software, use the release distributions!

In one project I went as far as making a "releases" branch, and actually putting the "make dist" output there, in full, and tagging that instead of the master branch. This was a pretty good compromise, since the github download link now points at the actual release tarball, and you can easily see from `gitk` which commit from the development branch each releases branches from.

However, it's annoying not to be able to use tags as a way of navigating the development branch, and since I wasn't the one officially putting together final releases, eventually it was found that this was just too complicated to keep doing for each release. And it's just a dumb trick to satisfy github's weird releases interface anyway, so I couldn't justify the complexity.

Re:GitHub responds to Dear GitHub letter by sytse on Friday February 12, 2016 @01:20PM (#11089204)
We have the same 'Source Code' link in GitLab on our tag/release page. Should we rename it into 'download source' to be more clear?
Re:GitHub responds to Dear GitHub letter by mdaniel on Friday February 12, 2016 @01:27PM (#11089271)
Or I am always a fan of putting a checkbox in the project's Settings page "show 'download source code' link" and a text box underneath it with "source code download label", enabling the project to use language they are happy with

That said, I am also cognizant that "checkbox overload" is a real thing, so merely "for your consideration"

Re:GitHub responds to Dear GitHub letter by sytse on Friday February 12, 2016 @09:54PM (#11092228)
We're indeed pretty hesitant to add options for these things, it makes it harder to refactor the UI if you have to account for all settings.
Re:GitHub responds to Dear GitHub letter by rogerbinns on Friday February 12, 2016 @11:02PM (#11092460)
Google Code used to have labels, in particular one named "featured". You can label files in the release with it, and the ui put those at the top. Consequently it was clear to users which release files would be best.

"Source Code" has two meanings. One is the preferred form of contributing to the project. The other is the preferred form for building and installing it on a system. Sometimes they are the same. However I argue someone going to a release page is more interested in the latter, and the former is confusion.

I'd suggest making your UI reflect the page visitor's goals. "Repository Archive" is better wording, but still not that good.

Re:GitHub responds to Dear GitHub letter by trevorrowe on Friday February 12, 2016 @05:41PM (#11091234)
The GitHub API allows you to provide much more useful content for a release. See this project as an example: https://github.com/aws/aws-sdk-ruby/releases - Notice that it has links for build artifacts, generated docs, and even contains links to issues closed by the release. Also, its pretty easy to automate a "release" from travis.
Re:GitHub responds to Dear GitHub letter by cdcarter on Saturday February 13, 2016 @12:16AM (#11092663)
The problem isn't that you can't add more content to a release, it's that you can't remove the misleading zipball/tarball links which are labelled source code but break traditional tarball conventions (e.g. not running autotools first).
Re:GitHub responds to Dear GitHub letter by grk on Friday February 12, 2016 @12:24PM (#11088764)
I'm not sure if that's what you're looking for, but you can edit release assets using Github's API: https://developer.github.com/v3/repos/releases/#list-assets-...
Re:GitHub responds to Dear GitHub letter by pravka on Friday February 12, 2016 @01:10PM (#11089118)
Absolutely. This feature is such an integral part of a repo that distributes releases that I'm surprised it hasn't gotten more attention from Github. I, for one, would love to see them provide the checksum for the archives that are published via Releases -- it's not currently available when you create a release and relies on the operator to implement a workaround that creates a digest and publishes it on an external endpoint.
Re:GitHub responds to Dear GitHub letter by mietek on Friday February 12, 2016 @12:58PM (#11089035)
Adding a script that builds the project from “maintainer state” and referencing it in the README may save you some aggravation.
Re:GitHub responds to Dear GitHub letter by rogerbinns on Sunday February 14, 2016 @12:41PM (#11099115)
Not really. Not many people have the whole autotools/automake tool chain installed, let alone document building tools, and whatever other processing the project does (eg SQLite uses some TCL scripts). So they'd have to go and get all those, obeying any version constraints. And after all that they'd end up at the same point as getting an archive intended by the maintainer for them to build and install.
Re:GitHub responds to Dear GitHub letter by mietek on Thursday February 25, 2016 @10:04AM (#11174846)
That is the intended outcome.