esjeon

Showerthoughts:

1. If you know ANN or, say, perceptron, you know you can eyeball models too. You can even have certain strategies in composing your own network. This weight-space programming thing is nothing new in that sense, and the article doesn’t really provide much beyond “LLM can do it”. That’s not even science…

2. I’m pretty sure this is Transformers thing, not even LLM. Transformer is, by itself, good at finding patterns in relatively-larger data. So this is unlikely limited to Astra.

3. This approach quickly becomes impractical as the scale goes up. Sure, you can also have multiple models chained with fewer connections, but, now that’s what we’ve been doing for years.

All in all, yeah, an interesting find, but too far from changing the game. Over-exaggerated every way. Perhaps one should start from formulating what weight-space programming is, what methodologies, practicality, etc. LLM bench is one of the last things you need.

Exactly. Either information flows freely, or everyone pays every bit of it. It’s clear which one is more beneficial to society. Barring distillation is an extraction of social wealth.

> LLMs will hit a wall in mathematical capability just beyond the convex hull of human mathematics.

I love that “convex hull” analogy. LLMs can certainly serve as a channel for knowledge transfer. They can help spread knowledge and know-how throughout society, enabling breakthroughs that were simply held back by practical limitations (e.g. lack of money, time, or expertise). Personally, I think LLM is more valuable as a tool for democratizing knowledge than as artificial intelligence.

I think the quality of Codex CLI (and Claude Code) is an honest reflection of the status quo of LLM-driven development. The future doesn’t look really promising.

It's no secret that people get blinded by their own belief and faith. They forget how to see, how to smell, how to taste, and how to hear. They get trapped in a virtual world inside their own minds -- a gigantic Matrix, but without sophisticated machinery.

Look at the current physical state of the city, the raw sight of it -- no amount of fair punishment can justify this scale of pure, one-sided destruction of lives, daily-lives, and futures. This is directed violence, clearly beyond practical necessity -- unless practicality is defined in terms of ethnic cleansing.

Any logic that jumps through the hoops of background material (history, religion, beliefs, politics, and so on) is mere mind tricks, designed to draw your eyes away from reality -- the horrific nature of the Israeli genocide.

(EDIT: grammar check)

Pin-pointing Hamas and Oct 7 is pretty much a logical trap, because Netanyahu and Hamas had been symbiotic -- which put them in the very same ballpark. Both enjoyed their own external threats, and, through that, established strong political influence.

The one difference is that Israel always had been able to control the situation for decades to its liking, while Hamas had been passively reaping the hatred that Israel had sown -- only until October 7.

So, Israelis fearing Hamas? That's a horror they created through their democracy, purely by abusing powerless people. Total hypocrisy.

Yup, exactly. The problem looks like this:

(1) OpenAI is merely abusing math problems for their PR stunt. That is, once the PR value dries up, they'll leave.

(2) OpenAI, and any AI companies, can incur serious harms to the social dynamics in the field of mathematics by taking away big prizes away from mathematicians.

(3) The current gen AI is too inefficient -- requires a lot of money to operate -- so it's near impossible for academia to replicate what OpenAI invested in Navier-Stokes.

(4) Also, human experts are still economically much more efficient in searching critical directions, as we saw in the case of Navier-Stokes. The best working model is human-AI collaboration, with AI taking over the grinding part. This is what OpenAI failed to recognize.

Thus, the aggressive campaign from OpenAI is socially harmful -- disrupting a working social system without offering any alternate working sustainable system.

An AI slop to filter out AI slops -- the pinnacle of AI.

Sadly this would not end just as a joke. We'll be doing this forever from now on.

> widen your reading, rather than trying to win a silly argument on a forum with 'logical fallacies' and other such humdrum wikiphilosophy.

Yeah, sure, the paragraph there is pretty bad, I admit. I was having an technical issue w/ my phone. But, look, did you really read my comment, the whole set of it? It's not even about philosophy. I'm just talking about a little /finding/ that Orwell's examples of bad-English is much better than what you usually find in your daily lives.

I'm not even arguing here, because you people avoid engaging w/ my point itself. It's more like I'm only repeating myself again and again and again. I'm not raising anything new.

> Your perspective on this is entirely wrong, starting with a mistake about the intentions of Orwell and the quality of the text (which is deliberately obtuse and plain awful in so many ways). If you can't easily interpret the biblical text, that's fine, it is quite old, but there is really no excuse for saying that Orwell is a bad writer

First, if you were talking about my first comment, I did mention that "I'm only guessing". I did clarify that I didn't read at that point. Someone replied with a link to the essay, so I read it, and only then I said "I [had] read it".

Second, I never said the original is bad. I only said the parody is pretty good, perhaps because of its brutal explicitness, which the first one lacks comparably.

Also, I've been saying that Orwell is a bad /bad-English/ writer, not a bad writer. His proficiency in English doesn't imply any proficiency in reproducing the bad-English he was criticizing. This part is childish, sure, but it was supposed to be /fun/.

> But Orwell's example here is an example of bad writing, not good, and was produced deliberately as an anti-pattern.

I totally agree here, but ...

> It has no redeeming features

... this is the part I disagree with.

Sure, on the surface, they are chokingly bad. I definitely agree that he removed certain qualities from those examples, and, for demonstration purposes, they serve the purpose.

However, if you offer those examples completely out-of-context, it's going to be difficult to dismiss them simply as bad writings. Their logical flow is very natural, and his word choices are very precise. It is much better /formulated/ than a lot of real-world bad English writings. I definitely sense /professional touches/ from those examples. That's why I said that it would take me hours, if not days, to write such sentences by myself.

In short: badly written, well formulated.

> we are all way worse at reading compared to the average mid-20th centry reader?

I would say English has changed. Modern English speakers, especially those in academia and technology, prefers explicit styles over poetic and nuanced ones. There are a lot of factors at play (e.g. i18n, the internet, education, literature) but I’m too lazy to cover them here.

> you - and no one else in the hundred years since - found his fatal mistake?

First, the essay is from 1946 — 80 years ago. Second, why not? Often-times, a different perspective is all that takes to discovers something new in the old.

I think you’re trying to make an appeal to authority here — a logical fallacy. You’re basically saying I cannot criticize his essay because Orwell is a great writer.

Also, my point is neither fatal nor about mistake. Orwell’s examples still demonstrate his claims in a relative sense, all within the context of the essay. However, when you pull them out of the context and put them out in the wild, it’s going to be very difficult to say those are bad writings right away, because, compared to other bad writings, those examples are very solid and straightforward.

> This is not even a sentence.

It is informal, sure, but I intentionally made it colloquial with something called bare quotation, a style commonly used in informal conversation. I wanted to avoid being “an overeducated dolt” and send a friendly gesture too, oh, but you chose to put that on a pitchfork. Bravo, clap clap.

No, my view is more like that he was too good at English to write bad English that he was criticizing. The examples he wrote are simply too sharp to fit well into his own criticisms. They are formulated so well that you can just “welp, my intention was different”.

EDIT: This kinda reminds me of the musical theory of harmony. There are strict rules to follow in music composition, but when you go against every single rule in major scale, and keep your work sound nice, you eventually get a minor scale piece that keeps every rule that you tried to break through.

Same here. He probably wrote a disgusting piece in his standards, but kept the writing sane, and eventually it became a proper prose — simply with disgusting intention in his standard.

That’s what I think happened here.

Ah, I just read through it. That’s was a fun read.

> This is a parody, but not a very gross one. …

So, Orwell wrote the second sentence as a parody of the first sentence, using the modern English that he was criticizing in the essay.

> … in the middle the concrete illustrations – race, battle, bread – dissolve into the vague phrase ‘success or failure in competitive activities’. …

> … The whole tendency of modern prose is away from concreteness. …

> … The second contains not a single fresh, arresting phrase, and in spite of its 90 syllables it gives only a shortened version of the meaning contained in the first.

He tried to argue that the second one is worse, but, unfortunately, he successfully simulated one possible interpretation of the original. Sure, the parody has narrower meaning, but it is a valid subset of the original meaning, which is still very significant tbh. This justifies the reduction of “race, battle, bread” into “success or failure”, invalidating one of his points.

Also, the parody is not necessarily less concrete. It replaced poetic expressions in the original with words that are bloated, for sure, yet more concrete. It does have awkward expressions, but every word plays a role in the sentence. This basically conflicts with one of his criticisms in the essay — meaningless words.

Funnily enough, his another example — “some comfortable English professor defending Russian totalitarianism” — also fails to capture the nature of the modern English. He points out the “euphemism” towards the violence as an issue of style, but, no, that’s the whole purpose of it, and his example really just excel at it. Style-wise, again, the writing is very solid.

All in all, Orwell simply failed to properly simulate what he was trying to criticize. Good examples of “modern English” can be found in the earlier part of the essay, basically written by other people. Those examples do fit into Orwell’s criticisms.

I think Orwell did a great job there actually.

Despite its bizarre look, the sentence is evocative and eloquent. It does make me get a clear mental image from the very first word. It leaves little room for roaming and guessing, as it firmly nails elements one by one, and, by the time I reach the end of the sentence, I get the full meaning almost immediately.

This sentence is not randomly written; this is crafted with intention. TBH, it would take me hours, if not days, to write a sentence this much condensed and easy to understand. I seriously like it.

Perhaps, this is more about context -- which style to use in which situation. I'm only guessing here, but, since Orwell is offering an interpretation, he probably chose to be more clinical. He probably had a point to make and didn't want to risk vagueness up-front.

Yeah, even Pangram is a bit problematic here. It has been notoriously fragile. Minor edits can flip scores from 100%-human to 100%-AI, because Pangram is crazy sensitive to local and surfacial features of text. Simple consulting with LLMs for word choices can result in 100%-AI score. Insane.

Obviously, NVIDIA is trying to own the AI development chain.

Owning HF -- the discovery and distribution channel -- is one thing, but I think the biggest threat vector is the privileged access to HF platform data, that includes HW survey info and model download pattern. This can be a borderline anti-trust case.

It doesn't answer at all. :/

One obvious loophole: upload your full outputs publicly. Anyone can feed those to their models.

Wait, we're already doing it. /s

Yes, but the token expires upon logout, and, after reading your comment, I just double-checked the expiry (got 400 from all related endpoints).

You simply don't have enough testcases.

I've just let Claude analyze an HAR file containing XHR traffic on the Facebook timeline. It happily wrote a script for extracting contents, fetch images and videos, and rebuilt the timeline into a single HTML -- highly responsive, with ZERO ads.

Now, parry this.

Lisp is a highly expressive language with little constraint, so there’s plenty of room for individuals to become strongly opinionated. That leads to a bit of elite mindset.

LLMs are only good at copying the surface, like frequently used words, sentence structures, use of metaphors, etc. They fail at copying the underlying thinking framework. LLMs cannot reproduce this, and they do notice the lack of framework in their writings (that’s how the know the writings are fake), but offer those as successful clones anyways.

I would call this one-row-per-contract-type, and this is the most general model for the problem (e.g. the model cannot be further broken down into finer level), thus, the most scalable model given storage is dirt cheap.

By saying coding is the hard part, you're insulting all engineers.

Edit: Although the current job market is heavily distorted, there used to be distinction b/w developer and engineer in the past. As the mainstream development model shifts from waterfall to iterative models, it became necessary for everyone to be engineering-capable -- only up to a certain point. So, every developer now carries a certain amount of engineering knowledge, but now they started to misunderstand and underestimate the value of engineering, and this is what you would get at the very end.

This one(Zapscape) exploits the same module(shadow MMU) as Januscape, so it does workaround the issue.

AFAIK the only code paths that activates shadow MMU are (1) lack of hardware EPT/NPT support (2) nested virtualization. Hiding `vmx`/`svm` prevents access to the second code path.

This one exploit "shadow MMU" in the nested virtualization path of KVM, so this one is more-limited than EPT/NPT vul'n (KVM defaults to EPT/NPT, nested virt'n is disabled by default).

Nested virtualization is rather a niche feature, and, tbh, considering that shadow MMU is highly complicated legacy code (i.e. outside of the main happy path) and has been source of critical vul'n, I would avoid nested virtualization on KVM.

> The model outputs are much more concise than when I try and talk to GPT-5.6 Sol about mathematics. By signalling expertise, Tao shunts the model into “talking-to-mathematicians” mode, not “explaining-to-amateurs” mode

I believe this works in two different ways.

First, information compression. The use of professional language helps describe problems more densely with minimal information loss/distortions. Verbose output by LLMs (e.g. ELI5) tend to incorporate local chat context, which can destabilize the context (e.g. out-of-topic, irrelevant nitpicking on writing style and wordings) and lead to faulty logic and even hallucination. LLMs are not good enough to look through all the noise, so, sometimes, it's helpful to refine the input data before performing actual tasks.

Second, boosting logical pattern-matching. Using professional language helps drive logical reasoning through simpler pattern-matching b/w texts. This is not about whether LLMs can reason or not; it's about how high-level reasoning is guided by preconception. Even humans tend to consume only textual surface of highly complicated theories (e.g. Adam Smith's "invisible hand"), and use them casually during conversation. It's similar for LLMs: if the conversation is conducted entirely in professional language, LLMs can easily incorporate external professional information into its reasoning. If the text is written in amateurish tongue, translating it into professional language can introduce errors and distortions.

So, yeah, keep your conversation professional, tidy and tight. A large volume of unprofessional text helps no one.

Most of the time, the domestic market in China is indirectly controlled through political and social influence, not through policy. It's not a fair market at all.

I think this kind of automated design process can un-hype humanoids. The point of humanoid robots is one-size-fits-all -- a generalist can readily fit into various scenarios. However, if custom designs are cheap and optimized, there's no reason to stick to inefficient generalist designs.

It's more like Claude models entirely suck at extracting key points. No matter how hard I emphasize that it needs to pick the "load-bearing" facts and claims, it cannot stop itself muttering around. It never nails the core logical structure. GPT is better at that.