defmacr0

That way it also works with third-party chatbots

Honestly, that's a weakness shared by a lot of human-generated science as well.

Bad module boundaries increase the complexity on both sides of the boundary

> Their primary purpose is to prevent lethal intergroup violence and blood feuds.

A state's primary purpose is to maintain a territorial monopoly on legitimate violence and to secure its continued existence. Generally this is at odds with its founding myth, but it's pretty obvious that states are also subject to a process that resembles evolution by natural selection. That said, I'm not even aware of any notable state or state-like entity that was created with the goal of "ending blood feuds". The vast majority of these territorial monopolies were created by means of conquest where the goal was to acquire resources owned by the people in some territory or with the goal of pooling resources to defend against such conquest.

"Only had a few years behind bars"

Yeah, in small examples it's pretty neat: Define a "sorted" property, write a law that checks that the list is sorted and contains the same elements and someone else implements the sort and together with a proof for correctness. However larger software systems have an exponentially larger surface for reasonable and unreasonable edge cases. When I hit ctrl+s in my editor I expect that my cursor does not change colour, that the window does not minimize, that the program does not crash if there is no disk space left and so on. I don't see how this does not devolve into "negative space programming" where the user would have to anticipate and constrain every possible unwanted behavior of his software.

This can easily lead you down a path of excessive modularity with each module being excessively complex though

They're almost certainly pin-pointing high-quality conversations and giving them a special weighting. Seems stupid to not do that.

Yeah and we get a nice list of attributions for who developed which idea, while OpenAI just takes credit for everything its model spits out.

A lot of math is extremely specialized, to the extent that only a handful of other experts in some field have any experience with those mathematical ideas, with most of them not even yet present in the published literature. It's really not a stretch to claim that it's pretty dubious when the AI decides to use these highly specialized tools after it has trained on chat logs where these techniques were being discussed.

Also, if we just take "high-quality" input data, which these chats would certainly be classified as, then the models are more than large enough to memorize everything verbatim. Spitballing some numbers, research literature suggests that LLMs are optimally trained with around 20 training tokens per parameter (fairly confident on this figure), that a DNN parameter encodes around 4 bits of data (less confident here) and I found sources in the 1-4 bits of information per token range (least confident here). So, fairly conservatively I would estimate that a model has the capacity to fully memorize around 5% of its training data, presumably high-quality data is a lot less than that.

If there's one thing I'm absolutely confident in, it's that Sam Altman personally goes to great lengths ensuring that ethical standards are upheld at his company.

> So why wasn't Anthropic behind their effort?

Presumably because this was something Levent did in his spare time and because it was not obvious that this work would eventually lead to a breakthrough.

> Why did Tristan use OpenAI's models when it should have been known was a potential outcome?

I'm sure in the past he had less cynical feelings about OpenAI and their penchant for academic fraud.

> I understand they wanted a normal math collaboration but presumably what Levent brought was his resources (as far as I can see Navier-Stokes is not his speciality)

I think you're not giving the guy enough credit in saying that his contribution came down to having an API key for Anthropic models.

> Normally these things are hashed out formally beforehand to avoid the sort of thing now happening.

How would that have helped? That agreement (which may well still exist) would not have involved OpenAI.

As a prior I would say that a math professor has about infinite times more integrity than OpenAI.

I am stunned anyone is giving OpenAI the benefit of the doubt

There's a spectrum between "barely scientific entertainment" and "dry technical reference". Also, it's not fully a zero-sum tradeoff, great authors have written serious textbooks that are quite entertaining to read, and there are pop-sci books that do a great job at covering advanced material.

I certainly would only trust my textbook as the authoritative source, but i can see that in the absence of an expert teacher it's nice to have something that can critique a proof. Imo the fact that it's hard to verify that your own proof is correct is one of the main barriers in self-studying math, especially if one is at a level where one is not completely fluent in applying the various techniques. This also applies to judging answers to open-ended questions in any other field.

Yeah, clean, neatly seperated and logically independent PRs are very nice for reviewers, but usually it requires one to complete the whole feature and then go back and think about the best way to seperate it into a series of smaller changes again. It works for projects like linux where there is tons of motivated manpower such that requiring authors put in a day of additional effort to make a change as presentable as possible is acceptable.

I take credit given all my infosec-related reddit posts they used for training.

Well at least it seems pretty implausible to me that a machine learning model trained to reproduce human text can in principle generate something that is significantly above human text production ability. If the "pea-sized semiconductor brain" is not a surprisingly shallow problem that you can just solve by interpolating existing research, I don't really see the LLM-approach to AI being the thing that makes something like it happen.

>we do use simulated evolution as a standard technique in machine learning.

Well, not really. For large-scale AI models it's almost exclusively some form of gradient-based non-linear optimization. Genetic algorithms (which is just hill-climbing optimization with extra steps) and genetic programming (which is really cool and not well-understood) do not perform all that well in practice and I'm not aware of any notable applications.

The article is "not even wrong". Most of the argument just circles around an inconsistent definition of what "real world" means.

> Remember "imaginary numbers" from algebra? You could do funky stuff with them like take the square root of negative nine. Imaginary numbers are generally useless for the physical world that we experience. However, they turned out to be very useful in the field of electronics. More specifically, they had predictive power in the field of electronics. Using imaginary number math, one can predict behavior of electrons and the accuracy of such predictions in models can be measured.

Like here, I struggle to understand what the point is. Is predicting the behavior of electronics "useless for the physical world we experience"? And that's ignoring the author's apparent ignorance of all the other ways in which complex numbers are useful in the physical world besides taking negative numbers' square roots.

> Any solution that can convince the Germans, the most privacy obsessed sticklers on the whole planet, has my support by proxy. If they think it's safe enough, it most likely is. Almost no other country has seen the dark side of what you're saying here as much as Germany, first with the Nazis and then in East Germany.

As a German citizen myself I argue that the more correct view of this is that the Germans are going back to their fascistic roots.

> Both Brexit and the Trump election have been significantly impacted by this, and it's not even controversial to observe that.

Yeah and most of that online chatter that made that happen originated from adult citizens that would not be blocked by online identification laws.

> Perhaps, but that doesn't mean that we should not address the elephant in the room - the seriously degrading impact that social media has on our society.

Well if you agree with my point then that would imply that foreign social media influence operations are not actually the elephant in the room. I think a convincing case that these measures would meaningfully improve the political climate has a much higher burden of proof.

> There was no debate because almost no one in (for example) tech circles is even acknowledging the problem, let alone coming up with a solution. Give me a better solution and I would argue for that instead. The status quo is unacceptable.

This "well we must do something" attitude where people are willing to accept anything that promises to solve some problem is how the worst laws are created. It's easy to forget that things can always be made worse too.

> Real life is not anonymous and consequence free either, why should online life be?

In real life you are largely anonymous if you go to any place that is not in your social circle. If I go to any physical store under the sun and pay in cash I remain completely anonymous. The only exception is staff checking your ID for alcohol etc. if you look underage, but that datum is only stored in one persons brain for as long as they bother to remember. Many people would take issue if the clerk noted every persons name when getting carded.

> what's so great about having complete online anonymity anyway?

Because liberal governments can always become repressive governments. Unless you have absolute faith that your government will never do something you have a moral objection to, you can never be sure that anything you are, believe or do will not be censored or land you in jail in the future. Infrastructure that has a minor benefit under a "good" government, but would serve as a major tool of repression under a tyrannical regime should not be built out of principle.

As a thought experiment, ignoring issues with technical feasibility, would you approve if every person is only allowed to leave the house, under a stiff prison sentence for violation, if they wear a shock collar that can be activated remotely by a law enforcement official? That way, if police want to arrest you, they don't need to use violence and they don't have to chase you if you tried to run. It would make the police's job a lot easier, would it not?

> If we can make sure that every real person can only have 1 social media account per platform, and if we can check that someone is an adult, and if (and only if) we can do that in a privacy preserving manner... then honestly, I don't see why I would be against that. I'm ok with being held accountable for what I do online

To the extent that political discourse is shaped by astroturfing, realistically it'll just give a monopoly of influence to whoever controls/bribes the company or entity doing the verification. There certainly would be technical/cryptographic solutions where there isn't some central entity with a master key, but I doubt that it'll work like that anywhere, especially if it requires a citizen safeguarding his own keys.

> I want to pay that price to prevent the severe outside interference we've seen in elections and in our politics.

Outside interference in the form of legal bribes (lobbying) and sometimes less legal forms of corruption has orders of magnitude more sway over politics than whatever the public may effect in elections.

> You and many others might not be, but it seems like you've lost the argument.

It's ridiculous to imply that there was any serious public debate on this.

> even when they'd benefit from blabbing.

Ok but what conspiracy theories are even filtered out by this? Whistleblowers almost always do it at great personal cost.

Formally politicians may be in charge, but at this point most political power derives from within the administrative state.

People, especially underage, commit suicide over private information of theirs getting leaked.

Also 95% of those 30k lines changed are fully self-contained inside of the aarch64 directory and of the remaining changes it looks like the majority is just adding "aarch64" as another item into an existing list. There are a few core changes that to me look like they could be done in their own PRs, but also core maintainers get to decide if they want to apply bureaucracy to their own work.

I don't understand that methodology in the first place. Does Anthropic even have some kind of somewhat objective definition to measure and judge "memorization"? Is there any evidence that other LLMs are viable tool to determine that?

If we were perfectly rational, then yeah, more data should never lead to worse decisions. However, it's easy to fall into the trap where being data-driven makes you only work on those things that you know how to measure.

That's worse than a report that can be easily dismissed