News for hackers. Stuff that matters.
Re:How to publish Git repos that cannot be republished to Gi... by Aachen on Monday November 02, 2020 @06:33PM (#24974723)
I read the post twice but don't quite get what the author wants to share. Is it some git(hub) trick I don't understand, just a way of saying Github's/Microsoft's monopoly is bad, or something else altogether?
Re:How to publish Git repos that cannot be republished to Gi... by renewiltord on Monday November 02, 2020 @06:38PM (#24974755)
Github complies with the DMCA, recently famously through the DMCA takedown of youtube-dl. The writer, Joey Hess (a relatively well-known OSS programmer, moreutils, for instance), is choosing to interpret this as their banning any repo that hosts the commit hash for youtube-dl (which they banned).

So he is jokingly showing you a way to create a Github poison pill. The joke is that if you include this 'illegal' hash you subject the repo to being DMCA'd out (not really, but it's sort of an 'imagine if'). Which means his repo is now unpublishable on Github (for long anyway).

It's kind of hacker humour in the sort of vein that you'd expect from an old-schooler like him. You sort of have to get the joke to get the joke.

Re:How to publish Git repos that cannot be republished to Gi... by Wowfunhappy on Monday November 02, 2020 @07:30PM (#24975068)
I'd find the joke funny if it actually worked.

Since as far as I can tell, Github isn't actually banning specific hashes (because they care about the content, not the hashes), and there's no sign that they're going to do so in the future, I can't say I understand the humor...

Re:How to publish Git repos that cannot be republished to Gi... by geofft on Monday November 02, 2020 @09:03PM (#24975648)
That was my initial reaction too, but thinking about it, it's potentially (probably not, but potentially) a violation of the ToS even if it's not automatically banned.

So, the hacker humor point is that you have a "poison" repo which, if you push it to GitHub, might get your account deleted in the future. You have no guarantee that it won't, for sure.

Re:How to publish Git repos that cannot be republished to Gi... by DarkWiiPlayer on Tuesday November 03, 2020 @02:54AM (#24977420)
The way I read it, it seems to claim that github will just "insert" a known object with that hash into the new repository when it's pushed, though I'm somewhat confused as to how this would work, or how the submodule trick even adds the hash to the commit log in the first place.

EDIT: I read the article again and couldn't find any of that, so I must have read that in a comment here on HN instead.

Re:How to publish Git repos that cannot be republished to Gi... by vSanjo on Monday November 02, 2020 @11:59PM (#24976474)
That was a very well-written comment, thank you. Hit the dopamine and gave me a smile for your explanation.
Re:How to publish Git repos that cannot be republished to Gi... by syshum on Monday November 02, 2020 @06:47PM (#24974824)
is it a joke though.....
Re:How to publish Git repos that cannot be republished to Gi... by o-__-o on Monday November 02, 2020 @06:51PM (#24974847)
..... yes
Re:How to publish Git repos that cannot be republished to Gi... by AceJohnny2 on Monday November 02, 2020 @07:19PM (#24974998)
It's a way to sneak in the hash of a commit (the latest youtube-DL commit) in a non-obvious way.

A git repo is fundamentally composed of 3 object types:

* Blob (effectively a file)

* Tree (which refers to collection of Blobs and Trees, effectively a dir)

* Commit (comment/ author info and various other metadata, a Tree, and a previous Commit)

This is what the git tools are used to dealing with. However git submodules changed things. With submodules, a Tree object can also contain a commit object! Together with the .gitmodules metadata file, this allows you to include another git repo inside your repo.

Joey leveraged that ability to add the youtube-dl latest master commit into his repo as a submodule, but deleted the .gitmodules file so that git wouldn't be verbose about it.

And that's how you sneak a commit into a repo.

This additionally leverages GitHub's data model (which is really the git data model), where they just have this huge DB of all the objects of all the repos on GitHub. So effectively by including this commit in a repo on GH, you're making it refer to the the entire (buried) youtube-dl repo, but you need to be sneaky to be able to see it.

Re:How to publish Git repos that cannot be republished to Gi... by _ikke_ on Tuesday November 03, 2020 @04:17AM (#24977900)
Note that it's only a _reference_ to a commit, not the commit object itself. So from a a github point of view, it would not even make sense to scan / trigger on this. As long as there are no commit objects with that hash, any reference is harmless.
Re:How to publish Git repos that cannot be republished to Gi... by Fellshard on Monday November 02, 2020 @06:36PM (#24974739)
They're using the fact the YouTube-DL (sp?) is being removed by GitHub automatically when rehosted in new repositories, due to DMCA complaint (and likely overreach). If someone rehosts his repository onto GitHub, the repository will be taken down, and the rehosting account likely banned.
Re:How to publish Git repos that cannot be republished to Gi... by akerl_ on Monday November 02, 2020 @06:41PM (#24974772)
Is there any evidence that GitHub is automatically removing reheated copies? Last I saw, GitHub had posted a comment asking folks not to re-host, at risk of being suspended, but there was no indication they’d started any kind of automated scanning.

Beyond that, it seems like a very community-hostile move to intentionally embed commit hashes that would even potentially cause a user to get banned from another platform if they push a copy of the code there. This doesn’t stick it to GitHub, it sticks it to the users of your repo.

Re:How to publish Git repos that cannot be republished to Gi... by viraptor on Monday November 02, 2020 @06:47PM (#24974818)
It depends if you want to be nice to everyone, it burn some bridges to make a point. The point being that there shouldn't be censoring of specific hashes.

Honestly if a user did get banned for sharing that poisoned repo, I would still put 95% responsibility for that on GitHub. But it's not happening yet as far as I know.

Re:How to publish Git repos that cannot be republished to Gi... by dx034 on Tuesday November 03, 2020 @01:35AM (#24976983)
It'll be a question of how many reposts they see. If enough people keep reposting with throwaway accounts, they will have no other choice but to start scanning automatically.
Re:How to publish Git repos that cannot be republished to Gi... by dheera on Monday November 02, 2020 @06:52PM (#24974858)
ROT13 the entire repo?

If they figure it out, then ROT14 it.

If they figure that out, encrypt it and post the key somewhere that blocks the offices of Github.

If they figure that out, we'll still find a way to build a better mouse.

Re:How to publish Git repos that cannot be republished to Gi... by nitrogen on Monday November 02, 2020 @07:09PM (#24974934)
If someone wants you to leave, it's usually best to leave. Good riddance to them if you find yourself in that position.

Far better would be to spend your efforts finding which RIAA members pressed the RIAA to make this takedown. Follow the money, cui bono, etc.

Re:How to publish Git repos that cannot be republished to Gi... by dheera on Monday November 02, 2020 @07:14PM (#24974963)
Why is it best to leave? Protests are a good thing, sometimes, and making sure it is on GitHub in some form or another is a form of such protest. At some point the relevant parties will have to realize that this mouse of a million engineers will always outsmart them. At some point they will have to realize that advocating for FOSS and DRM at the same time is hypocritical.

Rosa Parks was asked to leave, but she didn't leave.

Re:How to publish Git repos that cannot be republished to Gi... by leppr on Monday November 02, 2020 @09:29PM (#24975766)
People have been protesting against Facebook's policies almost since it was created (real-name only, no nudity even in classical art pieces, ...). That didn't change a thing. Facebook kept in-line with its policies, people stayed, and network effects only cemented the monopoly.

Github is extremely close to having a monopoly, leaving is the only way to prevent it from happening. If it becomes one, it has no incentives to listen to a few protests.

Re:How to publish Git repos that cannot be republished to Gi... by nitrogen on Monday November 02, 2020 @09:07PM (#24975666)
GitHub is not the instigator, as far as we know (MS is apparently a member of the RIAA though). I am as amused by the clever ways to get youtube-dl back onto GitHub as anyone else, but it's protesting the wrong thing in the wrong place.

The problem lies with whichever member organizations inside the RIAA (or nonmember orgs friendly with the RIAA) convinced them to file the takedown. Most of us aren't positioned to find that out, but that's where the most can be gained. Maybe the EFF or similar orgs can help. Then we'll know what the true motives were, and how to address them.

Re:How to publish Git repos that cannot be republished to Gi... by dheera on Monday November 02, 2020 @10:13PM (#24975953)
At the same time, technological measures can show it to them that they can't feasibly take it down and that they're fighting a losing battle. At some point the RIAA will just have to accept defeat because honestly it's not that much work to defeat their DMCA takedowns.

Right now they're issuing what, 1 takedown notice to GitHub? When we make that 100,000 takedown notices, some in ROT13, some in ROT24 on an Israeli server, some in RSA encryption on a Chinese server with keys published on Russian servers, maybe they'll just be too overwhelmed with paperwork at that point.

Re:How to publish Git repos that cannot be republished to Gi... by rrobukef on Tuesday November 03, 2020 @11:30AM (#24981114)
Add some honey-pots with unrelated content and counter-sue them for circumventing encryption on copyrighted content. Because how would they know what is encrypted unless they circumvented it without permission ... unless they perjured themselves?
Re:How to publish Git repos that cannot be republished to Gi... by netsharc on Monday November 02, 2020 @07:38PM (#24975117)
It's probably related to Google killing^W sunsetting Google Play Music and moving everyone to Youtube Music..
Re:How to publish Git repos that cannot be republished to Gi... by stavros on Monday November 02, 2020 @08:07PM (#24975296)
Joke's on them, I already ROT26 all my repos.
Re:How to publish Git repos that cannot be republished to Gi... by rrobukef on Tuesday November 03, 2020 @11:24AM (#24981045)
No, no. If they figure it out, you send THEM a DMCA notice for circumventing encryption to gain access to copyrighted content. Of course, you write your license (unencrypted) as open source except for specific companies.

Edit: On second thought, I just visited riaa.com . How about we lobby jQuery, WordPress and bootstrap to add a specific exclusion to their licensing? Now, this wouldn't affect the RIAA immediately, only the next time they update.

Re:How to publish Git repos that cannot be republished to Gi... by geofft on Monday November 02, 2020 @09:02PM (#24975639)
It's Git performance art/protest art, more or less.

Git has a mechanism called "submodules" which allows one repo to reference another repo. It doesn't actually include any of the content from the second repo: all that's included is the URL (in the file .gitmodules) and a commit hash (in Git's view of the directory structure).

When you clone a Git repository with submodules and you pass an option to git clone, or if you run a git submodule command later, Git will make a nested clone at whatever subdirectory path contains the submodule, and check out that commit. If you make commits in the submodule (and, hopefully, remember to push them), the outer repo will appear to be modified, and you can git add the changes, which will record the new commit hash, but only the hash.

The purpose is to deal with vendoring or sometimes making changes to large repositories owned by someone else without copying the whole source code into your own repository. (It's also occasionally used as a mechanism to split up very large repositories that are owned by the same group/organization, e.g., if you have a test suite or large data files or something, or a shared library used by multiple projects, you might find it helpful to use submodules.) https://github.com/ceph/ceph is an example repo that uses them - "ceph-object-corpus" and "ceph-erasure-code-corpus" at the top level are submodules, and if you click on the .gitmodules file, you'll find that there are other submodules inside src/, too. This is also an example of how GitHub handles submodules also hosted on GitHub - it will link you to the submodule at that commit.

So, GitHub does at least some parsing of submodules. The author is claiming that if you include a banned commit hash as a submodule in the history of your own project, and then promptly delete it, it won't noticeably increase the size of your Git repo, nor will you actually include any of the banned content yourself, but GitHub will potentially prevent that history from being pushed. As a result, you have a Git repo that GitHub would not accept.

Since the author both does not use GitHub and wants to encourage others to do the same, this would be a fairly effective way of forcing that.

(The article doesn't show GitHub actually refusing to accept the history containing this submodule, though - it only shows the submodule having been created, locally. Although maybe the author's argument is simply that pushing it violates the Terms of Service even if it's not blocked by automated means.)

Re:How to publish Git repos that cannot be republished to Gi... by BoorishBears on Monday November 02, 2020 @06:36PM (#24974742)
It's a trick to (theoretically) get someone banned for pushing a cloned project to Github despite not actually containing banned content.