inkysigma

I’ve seen quite a few critiques of the practicality of time complexity analysis but I’ve never seen the article describe what we should replace it with. Time complexity analysis as given has given some structure to the infinite zoo of problems even if imperfect and composes relatively nicely and is relatively simple. It’s also tractable, has mathematical structure, and is often practical as a heuristic (the kinds of algorithms people typically encounter or come up are typically reasonable under this kind of analysis). In order to be a proper substitute, something would have to satisfy all these qualities. Without such a substitute, we are back to analyzing problems and their complexity without any sort of framework which seems even worse than just remembering that sometimes the big O is not practical.

The only candidate that would be more practical that I know of is cache complexity for cache oblivious algorithms but that’s much more complicated.

I don’t really see how torrents solved this problem as so many seeders for any file could simply disappear. Not to mention, the significantly higher bandwidth costs of higher resolution video nowadays.

I’m less familiar with bilibili, but afaik it doesn’t do many of the revenue-sharing programs that YouTube does, so might have trouble attracting the same creators. They also have so far been pretty regional and so I’m not sure if we can use them as a sign of success yet.

That’s 2023? There’s a new series of accusations that the article is a response to?

https://www.wsj.com/world/ukraine-starlink-elon-musk-trump-d...

Well, the masking could be causal and I think that's been investigated by several groups.

Well, that’s true for a lot of TCS algorithms. The n log n algorithm prior was also not very practical for any numbers relevant to humans.

I also don't think there was general consensus on which way this would resolve prior to this (or is that a little out dated?) unlike some of the other major problem resolutions. I heard rumors that there would be a big result in TCS and speculation it would be UGC that or P neq PSPACE but I'm still a bit shocked.

I don't think this is how preferred suppliers have typically worked through the supply chain risk considering the US has definitely had preferred suppliers before and has not historically used this against a US company that was not on the list and the power of such a designation extends far past the military into all parts of the government. I'm pretty sure this has more traditionally been relegated to simple requirements or specifications.

I think this was discussed ad nauseum last time so I am wondering if somehow this has changed but the biggest part of the issue last time was the retaliation. You seem to suggest that this was due to the government not wanting restrictions on their actions. I feel this severely downplays how unusual this designation is considering, like Anthropic says, this has not been applied to domestic companies.

Why couldn't excluding Anthropic be done a different mechanism than a supply chain risk. Why wouldn't this be a standard part of an agreement or a request and simply refuse to renew or cancel a contract rather than being designated a supply chain risk. If some third party contractor for an unrelated non military reason wanted to use Claude as part of their process, it seems perfectly allowable.

If this was a remotely standard way of operation, why did a fair amount of corporate America sign briefs concerned with the retaliatory aspect.

Parts of the action seemed wholly retaliatory as well since Pentagon officials certainly used it as a threat. Why can't a US company have views contrary to the policies of the US government? Certainly the US government is free to not do business with them, but this designation affects everybody doing any kind of indirect business with the US government which is a rather long chain. If this becomes a legitimate mechanism, how might we distinguish caring about a secure supply chain and simply wiping out a company that disagreed with a pro war attitude? If a machining shop had a policy against manufacturing weapons at all, could they be blocked from making server racks for Microsoft or perhaps light fixtures for the Department of Labor? There are plenty of areas that are, again, non military? I don't think it's credible to claim that Anthropic will deliberately sabotage operations.

I think the Navier Stokes problem kind of illustrates what he’s highlighting. I think most people even before AI expected that this would resolve in the negative and that you could get finite time blow up. There wasn’t really ever going to be a situation where the resolution to this question, or really any of the other Millenium Prize problems as far as I know, gives some kind of immediate massive practical feedback.

The hope with many of these problems in math is that in trying to prove that, we get some additional insight into why it blew up that could be applied elsewhere to more general PDEs that cannot be easily controlled.

I think the observation from Tao and many others is that when humans solved these problems, the additional insights into intuition and theory building came for free since humans can give expository on what they found hard or what was their own intuition. This is much more difficult or tedious to extract from an AI model. Even when people did have access to the chain of thought, it wasn’t always very helpful to figure out what was the exact thing that made it all click. This is even more difficult how that the CoT are hidden but I would think the sort of difficulty of extracting the key ideas for a human might be worse now with more advanced models.

There’s a long term aspect to this too where we have historically used these problems as markers for the other parts of mathematics but if AI can solve it all, then suddenly this signal is not very meaningful.

Maybe to bring it closer to home. If an oracle just gave you P \neq NP, then this would be generally uninteresting since this was already expected. There’s a deeper question of why that needs to be answered. However, one would hope that creating such a separation would give us tools that allow us to create lower bounds on a lot more problems we do care about and perhaps some bigger insight onto what makes a problem intrinsically hard or easy. These long term considerations are helpful but are definitely more vague. The remarkable part is that AI is separating the part about proving theorems and the “free” insight you get.

To be quite clear, the solution to the _Navier Stokes problem_ is one in which you get a finite time blow up (i.e. infinite pressure). This is more meant to suggest that Navier Stokes is unphysical in some way which is not necessarily unexpected.

There's unlikely to be any engineering applications since even if the solution can be approximated, you still need to set up the initial conditions but at that point you can also drive pressure in other ways.

I'm curious what regulatory pressure there is on EU AI companies that wouldn't apply to many of the US firms. The AI Act and GDPR still apply to the tech giants since they operate in the EU (e.g. Anthropic watermarking). There are labor laws that are definitely more strict but many of the US firms have operating presence in the EU as well. I imagine that being directly in the EU probably does increase regulatory oversight but I'm curious how this would manifest in Mistral in particular.

Regarding local models, Gemma, GPT-OSS, Nemotron, and Inkling (maybe upcoming Muse series as well) all are fairly decent options at a variety of hardware costs.

This is rather weird framing since usually an additional citizenship (especially one in a country like the US) is considered a valuable thing to have for your child. Usually people claiming birthright citizenship are weird are doing it because they also see it as valuable. It’s also woven into the traditions of New World countries.

Regarding your concerns, I don’t see why it is unreasonable to simply renounce citizenship later or just simply double check your travel itinerary during the late stages of a pregnancy. This seems like at best an unfounded concern.

I know this is a bit cliche but I wonder how much headroom there is in the lower parameter count range. Is there any good reason to believe there is a lot of headroom or there is not? I suppose I'm just wondering if this wave of nearly Fable class models will be runnable on ~$10k worth of hardware at reasonable speeds in the near future.

I'm not the biggest fan of the current age verification schemes, but I don't think the idea kids can lie about their age differs in the threat model posed under either system. Presumably, under the law, parents would be the ones to create a child account with a non-editable-by-the-child-account age fields which would then be used as a legal source of reference. If you assume the child can create an account without such features or can edit the age themselves, then I don't see why the equivalent threat model in the parental control system would be giving the child control over the parental control toggles rendering it meaningless by simply disabling it.

That being said, age based restrictions isn't a fine grained control over the system as perhaps one would like but that also would be inherently more complicated to think about from a legislative perspective (e.g. how fine grained and how to categorize possible dangers) and user control perspective when it looks like a lot of parents are looking for a blunt generic button that basically goes "this is agreeable with general practices". This seems more or less how real systems are gated.

The other issue is that both present privacy challenges but this just a little more so from a fingerprinting perspective. Presumably you need quite a few bits to completely specify the filter whereas age is only a ~1.58 bit field in the CA model. Not really sure how much this matters when there are so many other signals for fingerprinting and we should probably make fingerprinting from it illegal but just some thought.

> Instead, what we get proposed is a system that cares very much about how old you are, and not one bit about the things that one's guardian understands one needs to be protected from.

Regarding your linked comment, I think it's a bit strange to say that if legislators really did care about child safety they would mandate fine grained controls instead. I'm not sure what additional fine grained factors you may be thinking of precisely, but we already use age as a gate in real life for many things we consider dangerous so it's quite natural for legislators to transpose those. Our laws already very much care about how old you are.

The final time complexity for Karatsubas does account for the addition via the master theorem and gives you something less than n^2. It’s just in some sense the recursion is dominant so we add to the exponent. As a result, I think the article is just talking about counting multiplications since it’s in some sense the “expensive” operation in the both the recursive karatsuba and the regular grade school method.

Given more code hosting services, I wonder if we'll also see a corresponding increase in the number of alternative VCS or if git is legitimately very entrenched as a tool. I am just being a bit grouchy but I do wish there was more development of alternative VCSs. pijul at least looks cool even if I don't know if it scales well. Git LFS can be somewhat finicky to work with so maybe we'll see perforce like systems. It's obviously not the most practical thing to have a variety of very different VCS's and definitely a PITA to learn multiple tools but git does seem somewhat suboptimal given the number of anecdotes about people just re-cloning the repo. I was recently trying jj and it seemed to work well (excluding the lack of LFS support) so here's hoping.

I actually agree to an extent with the idea that there is also some obvious political influence on LLMs on stuff like GLM or DeepSeek. This is reflected in conversations even on HN where this is brought up as a risk so I think this is somewhat accurate to my statement.

However, it's also unclear to me if this is directly coming from a directed political ideology from the firm itself or a more general "let's do what the government wants so as we can publish this stuff". Those imply two different ways about thinking of the model and whether we can sort of containerize the issue. I think if a firm like Huawei were to publish a model, these concerns would be significantly more vocal. For better or worse, many of these political questions are also distant to many users on this site.

On the other hand, many people on this website live in regions that are directly affected by Musk's constant political activism. It's hard not to be when he was such an active part of an administration that controls a global superpower and continues to push his view via X. The DeepSeek owners, by contrast, are not to my knowledge constantly calling for Taiwan to be invaded.

I do think if Musk was less politically active and less personally involved with his companies, there would be less discussion of Musk's politics. People, for better or worse, are willing to put aside political discussion, in the "everything is political" sense, that may be more loosely linked.

It is simply in the case of Musk that this tension boils over and legitimately becomes impossible. There is perhaps some kind of Singer-style argument about how this is some form of hypocrisy but as a practical matter, I don't think it's reasonable to ask people to turn down their political discussion around someone like Musk.

I think I understand your point but the funnier response to this question is that actually sometimes it is:

https://futurism.com/grok-looks-up-what-elon-musk-thinks

To your narrow point, it's very obvious that Musk influences the bot to share his views. For example,

https://www.nbcnews.com/tech/tech-news/elon-musks-ai-chatbot...

If your claim is that somehow I should not be concerned about Elon's politics with regards to the model itself, then this seems wrong.

Anyway, to the broader point of whether or not the we can avoid discussion about the Musk's politics and talk about the politics of the model as if it were independent of him, this also seems difficult. It is impossible to ignore because the man has made himself the face of every one of his companies and is an obviously political figure unlike any other company and has politics that are definitely characterized as more radical. This makes the political component basically impossible to ignore unlike any other company.

The next time the current American administration issues an executive order on AI, should the conversation always be limited to the technical merits of the executive order?

I think the distinction with the Chinese models (or with any of the other models) is that they aren't particularly vocal and obviously active about their politics. I don't see how political commentary about Musk's is somehow forbidden when the man constantly reminds everyone about his political position and is simultaneously the face of his companies and obvious beneficiary. Furthermore, he's also very obviously interfered with the model development in ways that are quite ridiculous compared to other labs to insert his political opinion.

I don't think you need to somehow get personally offended by every Tesla on the road but it seems ridiculous to ask people to not be political about a such obviously political figure.

This is essentially an open research question. ML theory is unfortunately very weak relative to where the empirics are. I think there's a relatively optimistic paper that was posted a while back here but I would also take it with a grain of salt.

https://arxiv.org/abs/2604.21691

There's of course empirical results and relatively weak theoretical results like the UAT but I also don't think that answers your question fully, especially since it seems impossible to definitively answer questions that the industry seems to betting on like whether or not there is a lower bound to their error rate or whether hallucination as a problem can be solved. We have much stronger ideas of what linear regression is doing relative to what LLMs are doing.

At a high level, the text samples are how the relationships are derived. If we treat text samples as sequences of tokens, then the sequences of tokens describe the joint distributions they occur together which confers the relationship between them. Iirc, this is related to the idea of the distributional hypothesis in NLP: the idea the semantics of words should be similar if they occur in similar situations.

One example along this path as an example is that every function must either terminate or have a side effect. I don't think one has bitten me yet but I could completely see how you accidentally write some kind of infinite loop or recursion and the function gets deleted. Also, bonus points for tail recursion so this bug might only show up with a higher optimization level if during debug nothing hit the infinite loop.

So I can't tell if the linked commit is an actual attempt or just an experiment but it did always strike me as odd to make a JS runtime in Zig when my impression was there were a lot of work-stopping compiler bugs at the time.

I'm curious how this view fits in with BERT or the T5 release which prior to the current LLM craze were the de facto language models for use in pretty much any tasks. Was this a position that would've otherwise grown without the llama release?

Some of those organizations (Linux and Mozilla) work on open source code for which they are already trained on. For clients like Apple, they almost surely have agreements to not do that.

Well it did contain a request to not notify according to that same letter. I suppose that brings up several questions.

1. Does that mean the same thing in the ToS?

2. How valid are these requests?

Depends on how legitimate you consider an administrative warrant and how willingly you think complying with one is.

On a more practical level, forcing them to go to court might not be much better. If this went to a FISA court, those are essentially rubber stamps and give nearly 100% approval.

His anti censorship stance isn't necessarily born out by the data:

https://www.washingtonpost.com/technology/2024/09/25/elon-mu...

X under Musk has sustained more government takedown requests.

https://www.washingtonpost.com/technology/2024/09/25/elon-mu...

I think in this context, scaffolds are generally the harness that surrounds the actual model. For example, any tools, ways to lay out tasks, or auto-critiquing methods.

I think there's quite a bit of variance in model performance depending on the scaffold so comparisons are always a bit murky.