News for hackers. Stuff that matters.
Re:Anthropic announces Claude 2.1 – 200k context, less refusals by swatcoder on Tuesday November 21, 2023 @12:14PM (#38366593)
Because they're ultimately training data simulators and not actually brilliant aritifical programmers, we can expect Microsoft-affiliated models like ChatGPT4 and beyond to have much stronger value for coding because they have unmediated access to GitHub content.

So it's most useful to look at other capabilities and opportunities when evaluating LLM's with a different heritage.

Not to say we shouldn't evaluate this one for coding or report our evaluations, but we shouldn't be surprised that it's not leading the pack on that particular use case.

Re:Anthropic announces Claude 2.1 – 200k context, less refusals by whimsicalism on Tuesday November 21, 2023 @12:24PM (#38366808)
idk we're just "have more kids" simulators and we do pretty good at programming as a side-task
Re:Anthropic announces Claude 2.1 – 200k context, less refusals by preommr on Tuesday November 21, 2023 @12:56PM (#38367275)
Someone doesn't get good at programming with low quality learning sources. Also, a poor comparison because models are not people - might as well complain about how NPCs in games behave because they fail at problems real people can solve.
Re:Anthropic announces Claude 2.1 – 200k context, less refusals by whimsicalism on Tuesday November 21, 2023 @01:12PM (#38367547)
We are both substrate that has been aggressively optimized for a task with a lot of side benefits. "NPC"s are not optimized at all, they are coded using symbolic rules/deterministic behavior.
Re:Anthropic announces Claude 2.1 – 200k context, less refusals by swatcoder on Tuesday November 21, 2023 @12:32PM (#38366850)
Sure, and those of us who have more robust preparation and expoure generally do a better job of it.
Re:Anthropic announces Claude 2.1 – 200k context, less refusals by YetAnotherNick on Tuesday November 21, 2023 @12:36PM (#38366898)
Github full (public) scrape is available to anyone. GPT-4 was trained before Microsoft deal so I don't think it is because of Github access. And GPT-4 is significantly better in everything compared to second best model for that field, not just coding.
Re:Anthropic announces Claude 2.1 – 200k context, less refusals by threeseed on Tuesday November 21, 2023 @03:49PM (#38369882)
And there is no evidence that Github is violating any open source licenses.

So they are going to be training on exactly the same data that is available to all.

Re:Anthropic announces Claude 2.1 – 200k context, less refusals by avita1 on Tuesday November 21, 2023 @05:23PM (#38371268)
Is this practically true? Yes, anyone can clone any repo from Github, but surely scraping all of Github would run into rate limits?

The terms and conditions say as much https://docs.github.com/en/site-policy/github-terms/github-t...

Re:Anthropic announces Claude 2.1 – 200k context, less refusals by vineyardmike on Tuesday November 21, 2023 @07:57PM (#38373093)
Well today you get to learn about the GitHub Archive project, which creates dumps of all GitHub data.

One example is the data hosted in Google Cloud.

https://cloud.google.com/blog/topics/public-datasets/github-...

Re:Anthropic announces Claude 2.1 – 200k context, less refusals by ironrabbit on Tuesday November 21, 2023 @02:51PM (#38369088)
Zero chance private github repos make it into openai training data, can you imagine the shitshow if GPT-4 started regurgitating your org's internal codebase?
Re:Anthropic announces Claude 2.1 – 200k context, less refusals by whimsicalism on Tuesday November 21, 2023 @03:46PM (#38369832)
You are downvoted but I agree.
Re:Anthropic announces Claude 2.1 – 200k context, less refusals by nomel on Tuesday November 21, 2023 @07:02PM (#38372478)
Org specific AI is, almost certainly, the killer app. This will have to be possible at some point, or OpenAI will be left in the dust.