News for hackers. Stuff that matters.
Re:NanoGPT by cs702 on Wednesday January 11, 2023 @07:24AM (#34337621)
Andrej doesn't need to do this.

He's done it because he evidently loves it, and wants to share his hard-earned knowledge with the rest of the world.

He may be a product of the ivory tower, but he's been in the trenches. He knows firsthand how f-ing hard it is to ship a product.

And here he is, sharing useful personal code with everyone.

This github repo has ~4K stars and its antecessor (minGPT) has ~11K stars. Normally, the number of people who clone or copy code from a repo is one to two orders of magnitude larger than the number of people who take the time to star it, so we can safely say that Andrej has helped at least a few hundred thousand -- but likely more than a million -- individuals around the world learn how to build and tinker with GPT models.

Remarkably, as I write this, no one else here has said thank you yet, so let me say it on everyone's behalf:

THANK YOU ANDREJ.

Re:NanoGPT by latexr on Wednesday January 11, 2023 @07:39AM (#34337713)
> Normally, the number of people who clone or copy code from a repo is one to two orders of magnitude larger than the number of people who take the time to star it

Intuitively, I’m having trouble believing that. Starring takes considerably less effort than cloning or copying code. The “time to star” is a literal second, maybe two if you have to scroll up.

From anecdotal observation, repos with more forks and/or external contributors than stars are far from the norm. I’ve seen many mentioning they star repos as a way of bookmarking they seldom go back to, or as an easy way to send kudos to the developer even when they don’t use the project.

In no way is this a comment on the value of Andrej’s work (I’m not familiar with it). I am only interested in the source of your “orders of magnitude” claim, which if proven will update my mental model of the coding community.

Re:NanoGPT by wongarsu on Wednesday January 11, 2023 @07:53AM (#34337836)
If I want to use a repository, my first step is to either download a released binary or clone the repository. Forking is much further down the line for me, when I've used the code, encountered a problem, fixed it, and decided to polish the fix up to make a PR. I star something when I either have used it and like it, or when I think I want to use it in the future and want to bookmark it (though the former more often than the latter). I have given out about 50% more stars than I've forked, and have probably cloned an order of magnitude more than I've forked or starred.

Of course not everyone is the same, but I'd be surprised if overall clones were less than an order of magnitude more than forks or stars, and find two or even three orders of magnitude believable depending on the target group of the repo.

Re:NanoGPT by cs702 on Wednesday January 11, 2023 @07:57AM (#34337865)
Exactly. I would add that the number of clones (not forks) and file/page views is viewable only by the owner of the repo, so we can only guess. (If you own a github repo, you can see the most recent number of clones and page views by clicking on insight -> traffic.)

My estimate of "one to two orders of magnitude" is based on anecdotal evidence. I edited my comment to reflect as much.

Re:NanoGPT by sjadoinqwoeihad on Wednesday January 11, 2023 @08:05AM (#34337930)
I checked my 5 year old repository of ~300 stars. It has a ~100 unique clones a month. So if the average was half of it then the 1 order of magnitude would be quite an accurate approximation.

I think the biggest difference with a clone and a star is that a star requires an account and some vested interest in the social network of Github. Anyone who is not interested in the social aspect can just bookmark it.

I guess this differs quite a lot by target demographic. A tool for GPT will probably get a lot more stars than a plugin for some consumer software simply because it is more targeted for the audience of people who have Github accounts.

Re:NanoGPT by cs702 on Wednesday January 11, 2023 @08:16AM (#34338025)
Thank you for sharing your anecdata. In my experience, the number of clones per month is much higher at first, and then decays gradually until it settles into a stable run-rate, so it's likely that you've had more than 100 x 12 x 5 clones over those five years -- i.e., between one and two orders of magnitude the number of stars, 300.
Re:NanoGPT by jefftk on Wednesday January 11, 2023 @09:15AM (#34338692)
Another data point: icdiff is 13y old with 4k stars and 200 unique clones in the past month.

(This is a tool that most people install and run without any interaction with GitHub, since it is in package managers)

Re:NanoGPT by latexr on Wednesday January 11, 2023 @08:25AM (#34338109)
> I think the biggest difference with a clone and a star is that a star requires an account and some vested interest in the social network of Github.

Agreed, but that also means you may be getting multiple clones from the same person/organisation, which artificially inflates the number.

Re:NanoGPT by chirau on Wednesday January 11, 2023 @08:22AM (#34338089)
How many projects have you starred and how many have you cloned?

Whilst starring is simpler, the incentive is much lower than that of cloning. Especially for projects you just want to use and not contribute to or follow.

In my many years of work, i have only starred less than 50 repos. I am sure i have cloned more than a thousand.

Re:NanoGPT by latexr on Wednesday January 11, 2023 @08:36AM (#34338232)
> How many projects have you starred and how many have you cloned?

I seldom star, but neither you nor I can be extrapolated to the general community. I have thousands of stars in some repos, and I know a significant number of those users don’t even code, let alone clone repos or copy code, they’re interested in the final product. They have GitHub accounts because it’s the way to report bugs or make feature requests.

The OP made a claim. All I’m looking to understand is if it has data backing it up or it’s just a gut feeling, because if it’s the former I’ll have learned something and made a correction of my mental model of the world. Sharing more anecdotes will leave us stuck in the same situation.

Re:NanoGPT by londons_explore on Wednesday January 11, 2023 @08:34AM (#34338209)
Some repos have code that 'phones home' when run. For example, checking for updates or security vulnerabilities.

By checking the usage statistics on that server, you can get an idea how many users there are, and typically it's far higher than the number of stars.

Re:NanoGPT by latexr on Wednesday January 11, 2023 @08:44AM (#34338313)
That just tells us that more people use the code than star the repo. I don’t think that’d be a surprise to anyone. The claim was that more people clone and copy code from the repo than the ones who star it, which is a different matter from the number of users.
Re:NanoGPT by cs702 on Wednesday January 11, 2023 @08:48AM (#34338374)
Thank you for clarifying. I meant use. The number of clones and the number of file/page views are proxies for that. So is the number of installs via pip, conda, and other Python package management systems, in this case. I updated my comment to reflect as much.
Re:NanoGPT by hahamrfunnyguy on Wednesday January 11, 2023 @10:12AM (#34339282)
I've stared maybe 2-3 repositories over the past 15 years, contributed to probably a half dozen and used hundreds (if not more) in my applications. To me using means using that project in an application you develop. Typically I get them from NPM or Nuget and I contribute when a) the project owner thinks my feature idea is a good idea or b) I run into a bug that I can fix.

Starring is just not that useful to me so I can see why users or contributors would be much higher. I typically star repos if it's an unpopular or old repository that doesn't have NPM or Nuget packages.

Re:NanoGPT by tomComb on Wednesday January 11, 2023 @08:42AM (#34338299)
Him doing this is not like when your average bloke does it.

He appears to be building a business and maintaining his profile. And there is nothing wrong with that - I admire him for for pursuing his career in this positive and helpful way.

But random folks do this sort of thing everyday with no such career goals and little recognition, so I'm not sure it is this specific contribution that needs to be called out.

Re:NanoGPT by krisoft on Wednesday January 11, 2023 @09:10AM (#34338633)
> I'm not sure it is this specific contribution that needs to be called out.

I go the other way. I would like to thank anyone who releases open source code, whether they cause big ripples or not.

Re:NanoGPT by modeless on Wednesday January 11, 2023 @11:26AM (#34340238)
What business is he building?
Re:NanoGPT by adam_arthur on Wednesday January 11, 2023 @10:52AM (#34339795)
I’m all for thanking open source contributors, but your excessively prostrating wording is a bit much for me.
[dead]
Re:NanoGPT by idiotsecant on Wednesday January 11, 2023 @12:22PM (#34341171)
can't tell if you're making some kind of clever quip or if this is some random spambot just entering a random reverse DNS lookup line.
Re:NanoGPT by cs702 on Wednesday January 11, 2023 @01:23PM (#34342276)
If I overdid it, I'm sorry. I promise it wasn't intentional. My comment was a spur-of-the-moment, motivated by sincere gratitude :-)
Re:NanoGPT by mrg3_2013 on Wednesday January 11, 2023 @11:42AM (#34340497)
Thoughtful post! Everything so true! I am always amazed by individuals who truly are educators of the world.
Re:NanoGPT by isoprophlex on Wednesday January 11, 2023 @12:43PM (#34341548)
Pedantry time!

A million people building GPT models means that one in 8000 humans on earth has built one. That seems wildly off.

Linkedin has about 100.000 profiles of data scientists. Assume generously that the actual number is 10x higher. Not correcting for the fact that a data scientist isn't always a machine learning expert, etc etc, there's just no way every single one of them even KNOWS what a GPT-like model is.

Re:NanoGPT by cs702 on Wednesday January 11, 2023 @01:21PM (#34342239)
Not only building. Also tinkering, using, testing out of curiosity, etc. There are around ~30 million software developers worldwide today (source: googled it). Around ~7 million of them are registered users of Github (source: googled it). 1M+ seems likely to me.

BTW, I appreciate that you preceded your comment with "Pedantry time!" -- nice gesture :-)

Re:NanoGPT by LeoPanthera on Wednesday January 11, 2023 @12:53PM (#34341734)
I love italics. They're good.
Re:NanoGPT by cs702 on Wednesday January 11, 2023 @01:26PM (#34342324)
In hindsight, yes, I may have overused them out of excitement. Sorry! :-)