The most realistic one is, that people will get fed up and register an account, and github can brag about how many new users they brought in.

The other side is, that existing users will get pissed at github, because they can't even search anymore without logging in, and sometimes that's a pain (not their pc, public pc, incognito tab, the time needed to do the 2fa, etc.).

Github can still keep the cheap, fast basic search for users not logged in, but they didn't.

Keeping the old search running for logged out users has substantial costs too:

1: You have to maintain two full independent search and indexing implementations

2: User confusion. Why are the search results different depending on if I'm logged in or logged out?

I agree with you, but I also feel like you're throwing pearls at swine. Some people will always have an incredible amount of entitlement to justify their laziness or incompetence.

I mean, the person opening the issue escalated this into socioeconomic issues just because they don't know how to use grep... What else is there to say?

> I mean, the person opening the issue escalated this into socioeconomic issues just because they don't know how to use grep... What else is there to say?

I suppose one could try to see the value in the argument and engage in good faith discussion but it's easier to flippantly dismiss people like this, I imagine.

Sometimes you have to be flippant when engaging with trolls.

There wasn't enough thought behind the issue, other than being outraged at their unsatisfied entitlement

> There wasn't enough thought behind the issue

Slightly more than your response, at any rate

In reality, how many people are regularly using github search without having a github account? Can this change really be expected to bring in a meaningful number of users?

Trying to think this through, my own best guess for what is going on is that there is some amount of traffic coming from bots/scripts and github would like to have all queries associated with an account so that they can block accounts? I'm not sure if this makes sense, but I think it's a reasonable possibility.

most open source projects migrated to gh when they pledged it would always be free and open for open source...

now under new ownership they closed search and "pray they don't alter the terms further"

ps: also everyone here guessing wrong. Microsoft is an advertising company. requiring you profile for a service is self explanatory.

Is it really an advertising company?

Seems to be around 5pc revenue from a quick search.

Not that I want to defend Microsoft.

I’m not saying you’re wrong, if anything it makes their decision even more ridiculous, but GitHub has completely saturated the market. Other than pissing people off what do they really benefit from this?

> GitHub has completely saturated the market

That's an assumption which github may not agree with.

This irks me to no end.

They justify it with "engagement" and it kinda works to a point, since if you don't get pissed with the broken search and register an account, you're not considered a "user", and if you try to use the tool but don't want to register you're not "engaging" with it, I guess... But it makes the initial contact that much more miserable.

> people will get fed up and register an account, and github can brag about how many new users they brought in

This makes no sense. The number of people who use GitHub code search but don’t already have a GitHub account is surely negligible

it's about having you logged in for the day.

logged in user ad impression is 8 to 12 usd. anonymous user is .01cent per thousands.

I don’t see how that relates to my comment

I believe the previous poster means that there are people who have an account but aren't logged in all the time.

I honestly don’t understand why. On my computer I log in once and that’s it. They don’t automatically log you out. I haven’t logged into GitHub since setting up my computer.

There are ads on GitHub?

o wow, the KPI of user acquisition should be through the roof with login required

If you really care about being able to search github without logging in https://grep.app/ is pretty good, I would often use it instead of the old github search cause I found the results to be better.

the most realistic one is to prevent other scrapers from building their own tools on top of github code search.

by putting login, they can throttle or block it entirely.

That's interesting but the technical reasons are really besides the point.

The end result is that it's no longer possible to participate in "GitHub hosted open source" without actually being a member of GitHub.

Agreed. If this is indeed a resource issue and they want to just block crawlers/bots/anonymous users from using lots of resources then perhaps having the old "less resource intense search" for users who aren't logged in and having the new "more resource intense search" for users who are logged in would be an improvement.

Ideally yes but the old code search may also require costly infra, like building specialized indexes of the codebase (eg an elasticsearch index).

Since those are "fixed costs" per repo rather than per search, they'd now be much more expensive per search if they were only used by logged-out users.

Pure outside speculation of course.

You can still clone and grep, can't you? You can't file issues or submit PRs through Github without an account, either.

You're kind of making my point, one needs to take their code elsewhere to be able to browse it.

In other words, GitHub is no longer suitable to host open source repositories because of all the closed "extras" it adds on top.

How does providing more features to some people make the core service unusable for "open source"?

This is adding less features for libre software users. I hope everyone migrates to local SourceHut servers when possible.

Are you suggesting that there's a place that you can file a PR (or equivalent) without an account?

They're shady AF, but some features require an account not because they're moat water but for some other reason.

I'm really focusing on the search feature that's no longer available for a given repository unless you're logged in. Everything else is "Microsoft added extras to git". If all I can do on GitHub is use it to git clone, then what's the purpose of it?

It's clear the "extinguish" phase of Microsoft's open source venture has started. There're plenty of very good alternatives, it's time to move on.

There is actually such a place! The Linux kernel (the original use case for Git) is developed by emailing patches to the appropriate mailing list. The mailing list is compatible with self-hosted mail systems.

But you still need an account. email one in this case.

You don't need an account to read the list.

http://vger.kernel.org/vger-lists.html

Unless you count your ISP of course.

You don't need an account to browse GitHub PRs or issues either.

You need an e-mail for GitHub as well, but in addition a GitHub account

You don't need an account to send email. You can send emails using nothing but a CLI tool on a *nix machine. Although most mainstream email services will send those to the spam folder these days.

I thought damn near every ISP blocks port 25 outgoing. I even had to open a ticket to get it unblocked on a vps I use.

probably for the best.

Do you mean "git clone"? I thought the idea about git is that you always take the code into your local machine before doing anything. Sure, github gives some bells and whistles to obviate the need from time to time, but that's basically decorations.

And can you search a git repo on git://my-domain.example.com? Or can you browse that without cloning? And even cgit, can you search that?

Are these now also no longer "open source"?

No.

None of this is somehow a core requirement of "open source".

You can browse code on Github just fine (and search within a file!) - you just can't search through the tree, which is a function of whatever platform the code lives on, be it your box or Github or some other web UI.

Its never been possible to participate in github hosted open source without an account...

I have a GitHub account. Until recently most of my GitHub participation was done without being logged in.

What do you do that doesn't require authentication?

Read code and issues mostly.

It's rather pedantic, but I wouldn't call that participating. That's more in the realm of observation.

An account has always been required for active participation, e.g. committing directly to any repo, submitting a PR, opening an issue, etc.

Yes, and I use an account for those. But I value not having to use an account for any work before that, which is just a crucial.

You can still do that without being logged in, thats also not participating in open source. More observing open source then anything

These are prerequisites to what I suspect you have in mind when you consider open source participation. Of which I do a decent amount, logged in of course, but this comes first.

right, but being a member isn't a pay-wall. This feels more like an academic argument than a practical one.

Becoming a member means sharing your personal information with GitHub and abiding by their terms of service. It also implies that they will then track and profile locations where you sign in, technologies you use to interact with their services etc. That's a lot to give for "I want to browse a piece of code someone made for free".

The only bit of "personal information" you need to provide is an email, and you're free to use a burner.

> That's a lot to give for "I want to browse a piece of code someone made for free".

You can still do this. All repositories are accessible without signing in.

And it's what they get in return for hosting and providing indexing and a git server.

I'm not saying it's a deal that makes sense for everyone. But it's certainly a barter trade and is well within their prerogative to offer.

Facebook recently estimated that knowing about a user costs close to €11/month. That's a lot of money for hosting and indexing git repositories... I guess we need an alternative.

And it's EXACTLY the same on Codeberg, and GitLab, and SourceHut, and anything else online. Some of these don't even have a "search" feature.

This is just degrading in to conspiratorial FUD.

Giving them your personal information is a form of payment. I don't mean this in a pedantic or academic sense, but a very real one.

Not at all. FOSS should be built on FOSS tools.

The kinds of people averse to user accounts wouldn't be using ("participating in") Github in the first place.

Or to put it another way: It's a surprise you need a Github account to use ("participate in") Github?

I see lots of reasons being proposed, but my guess is that they want or need to avoid content being scraped by AI related companies and researchers.

It's not their code to "protect". The license of each project dictates what can and can't be done with it.

And also, what is stopping the scrapers from creating a bunch of accounts to scrape with, and rotating them once they get banned?

As someone who contributes a decent amount of OSS code on GitHub, I want my code scraped by AI related companies. If GitHub starts making opinionated decisions about how to “protect” my code, then I won’t like that.

> If GitHub starts making opinionated decisions about how to “protect” my code . . .

They already did.

> . . . then I won’t like that.

There is no universe in the multiverse where Microsoft or one of its subsidiaries gives a single flying fuck. Something about those who forget history being doomed to repeat it . . . .

A scraper won't be using search. Only crawling links. If they required login to browse a repository that would stop bots but also cause a 1000x greater riot.

I just assume this to be the case for all future decisions like this. Surely the Reddit API changes were at least partly from this? Reddit is one of the richest sources of (pretty) authentic human interaction that humanity has ever made. Every AI company and startup was going to mercilessly mine this forever. I'm sure Reddit would much rather sell (our) interactions instead.

Does that mean LLMs are going to speak like redditors? Oh the humanity.

They already do. :)

https://www.reddit.com/r/ChatGPT/comments/10j531u/i_made_cha...

If the new code search is more resource intensive, then why not have a slimmed down, less resource intensive version for when users aren't logged in?

This may involve running two different sets of (for example, ElasticSearch) clusters, which doesn’t justify itself from a business perspective, especially given that non-logged in users can still clone code and use local tooling for search.

As far as their code search under the hood it's just Elasticsearch, nothing special. Github's explanation sounds like mostly bullshit. Forced authentication of public endpoints is not an appropriate solution for the issue they're claiming. People building bots who actually care about this endpoint can just have their bots login with free accounts. This doesn't actually prevent load on their servers in a meaningful way.

When they’re logged in, they are much easier to throttle or even ban compared to IP hopping to dodge IP throttling/bans.

>bots login with free accounts

those accounts still need to be created, which is hard to automate

also, now GH can just ban more easily detectable accounts that scrape, vs anonymous visitors

GitHub's code search is no longer using Elasticsearch, it is using an entirely in house (Rust-based) search engine nicknamed Blackbird: https://github.blog/2023-02-06-the-technology-behind-githubs...

Might also help with thwarting easy access to tokens/keys... especially for those that are not currently in their filtered list.

While a bad actor can certainly scrape/clone the same data... Given repos with thousands of files... Scanning by scraping vs searching vs cloning.... Search is certainly the cheapest for the attacker while the most expensive for GitHub

That could be mostly solved by requiring login only for searches across all repos. Searches in a single repo could be anonymous.

In fact, that's exactly how it was until in June GitHub started requiring login for searches even in just a single repo.

You'd think, with all that work going on under the hood, that they'd be able to support searches for repos by multiple paths:

> Give me repos with project.toml and a flake.nix at the root

The interface supports it, but searches come back empty. I end up leaving scripts running while I sleep which walk search results and narrow them by just checking to see if the files are there.

If you're out there Microsoft, please fix your search so I can stop being a bad citizen.

The new search is still so useless that I just maintain a full local clone of our org's code. Someone was requesting perms to run a script to search across the org today -- it would've taken 10+ minutes to run while respecting undocumented secondary rate limits (which there's no way to check via headers, by the way). I rewrote it to use ripgrep locally, and it's down to 20 seconds.

It's likely to avoid crawling, especially to train competitors to GitHub Copilot or other LLMs.

Why can’t you train on the git repository itself? Clones are not behind a login.

The way AIs are evolving, I wouldn’t be surprised if they have the ability to go to GitHub and search on demand based on a users query.

How do you discover the repository, especially in an automated fashion?

This seems like the most probable reason of any I’ve read so far.

Aside from performance concerns, there have been many occurrences of bad actors using code search to find hardcoded credentials, and then using that to gain unintended access to additional systems.

I suspect the change has more to do with security concerns (and having an audit trail) than performance.

I’d like to see these many occurrences.

I think the solution to hard coded credentials isn’t making search harder (security through obscurity == bad) but the other things GitHub is doing to detect and mitigate hard coded credentials.

Search is still possible using google and other methods, so any theoretical gains from forcing login are dangerous to count on as vulnerable projects are still vulnerable.

If anything, I think the solution to projects with hard coded credentials are to make the credentials easier to find and exploit so they fixed more quickly after being created. The most dangerous are ones they are hard to find so someone uses them for long periods of time without detection.

> I’d like to see these many occurrences.

Some high-profile cases where credentials were leaked on public GitHub: Uber in 2014 and 2021 [1, 2] and Twitch in 2021.

> Search is still possible using google and other methods

Yes, you can search with Google or other sources. But the thing is, pretty much only GitHub has easy access to all the code present there, readily available to search within minutes of pushing.

You could try mirroring GitHub yourself, but you'd need enormous disk space and bandwidth, and would quickly hit rate limits. You also wouldn't be able to do fast full regex search like you can with GitHub's search, as you don't have their search infrastructure.

Hackers are aware of this and do make use of GitHub's search to identify possible leaked credentials. I recently experimented with uploading a dummy AWS credential pair to a public Git repo, and saw that numerous IPs started trying those credentials less than 5 minutes after I pushed.

> any theoretical gains from forcing login are dangerous to count on as vulnerable projects are still vulnerable

Indeed, requiring login is not going to _solve_ the problem. But it could lessen the impact of leaked credentials at large scale, by making it more difficult for automated systems to harvest them.

Perhaps more significantly, requiring login could give better audit trails in incidence response situations, as the logs would indicate which accounts were searching for secrets.

> I think the solution to projects with hard coded credentials are to make the credentials easier to find and exploit so they fixed more quickly after being created

Yes, this can help! There are several other companies that specialize in secret detection. I've also written Nosey Parker, a fast regex-based detection tool that has higher-precision rules than similar tools [4]. GitHub also has its own offering in Advanced Security to address this problem.

[1] https://www.reuters.com/article/uk-uber-tech-lyft-hacking-ex... [2] https://www.securonix.com/blog/securonix-threat-research-ube... [3] https://news.ycombinator.com/item?id=28770590 [4] https://github.com/praetorian-inc/noseyparker

Bad actors are hardly deterred by the requirement to be logged in to be able to search.

Yes, you are correct. But requiring that one be logged in to use the search functionality makes a stronger audit trail possible: looking at the search logs would indicate who has been hunting for secrets!

If that's the case, they should really fall back on a grep of the codebase because whatever is "under the hood" sucks. It routinely returns no results for exact string matches in code. These days I clone the repo and use ripgrep instead of codesearch because it's worse than useless, it's wrong.

We have been running Internet without requiring user accounts. Search was possible. Has something changed? Has github been having real problems with bots? No?

why cant they just use the old approach for anyone and the good stuff for loggedin users tho.

If that's true then they should still provide the old non-heavy search for non-logged in users.

Check out the Chromium code search. It has regex support, and doesn't require login. Although it is smaller than all of GitHub.

Disclosure: I work at Google.

https://source.chromium.org/chromium

Probably code search was misused by bots to search secrets.

Yeah but that’s been the case since search existed. It seems unlikely that somehow now that’s become an urgent problem or requiring login will fix anything.

is there any reason why can't GitHub put their code search behind something like Cloudflare Turnstile? if HaveIBeenPwned can do this, why not GitHub? https://www.troyhunt.com/fighting-api-bots-with-cloudflares-...

I suppose maintaining two systems adds overhead too, but they could've let you use "basic search" without login, with the improvements login-gated.

If they are doing this intentionally, then fuck Microsoft. Bunch of assholes as always. If this is mere incompetence, then again I'm not surprised that they fumbled such a simple task. Grep is literally 50 years old. They don't even have to write any code to do text search, they could use free software if they love it so much. This isn't some amateur startup with a CRUD app and 3 devs. This is one of the largest and wealthiest tech companies in the world, and they can't even get text search to work without a login? What a bunch of losers.

If there is a 3rd explanation, I'd love to hear it.

> If they are doing this intentionally, then fuck Microsoft. Bunch of assholes as always.

They explicitly said that they "had" to require being logged in, so, yeah: https://github.com/orgs/community/discussions/77046#discussi... . Although they do assure us, however, that they're totally sorry for the inconvenience.

You are correct, but I was just covering all the possibilities since I never trust what they say either.

Where in that post does it say that the user not being logged in is making it difficult/impossible for them to return search results?

They were already rate limiting unauthenticated users.

So are expected to believe that a hostile user who has zillion of ip addresses (to get around the rate limits) won't also be able to make a handful of accounts?