prngl

There are 2 economic arguments for why it would make sense for Chinese State to subsidize the open-sourcing of models beyond undermining Anthropic's and OpenAI's investments (and by proxy the American financial economy, ie capital class):

1. As induced demand for domestic semiconductor production, where the level and diversity (ie number of distinct corporate users) of demand for the hardware is tied to the availability of models you can run yourself, ie open-weight models. If you believe that semiconductors will continue to be an important sector for innovation, productivity growth, and security, then it would make sense to subsidize broadly now, for future gains later. This would be the same export-led manufacturing discipline that allowed China to successfully develop several other sectors over the last 50 years.

2. It is likely that the bulk of value production will happen above (and below, ie #1) the large models. We already know that 90% of the training cost (maybe even closer to 99%) is in the single pre-training, but that an enormous amount of the value is actually in the supervised, RL, constitutional fine-tuning, and harness building that happens afterward. So, if your interest was in maximizing the size of the pie, you may actively subsidize the pre-training so as to maximize the downstream usages. This induces a direct value transfer from the labs specializing in pre-training to all downstream builders and users. There's a similar logic to subsidizing or state-financing the construction of other infrastructure and basic research.

This was an interesting interview. Like a lot of great comedians, Odenkirk has a very grounded and bleak view of the world. I suppose a lot of art, comedy included, is a way of coping with their perspective, for themselves and for the audience.

Should have specified old used thinkpads. I’ve never bought one new. My daily driver is 10+yo, bought for $200 and upgraded mem, battery, and ssd with another $100.

Resonates. Reminds me of old Thinkpads. Cheap sometimes means accessible, simple, minimal, functional.

Very Don Norman problem. It is odd that things like this aren't more of a "solved problem." Maybe they are in some places and builders in the US just ignore the best practices.

These are the points I'm making, which I think are fairly one-to-one with my original comment:

- American politics presents a false choice between Democrats and Republicans

- America is both a consumerist and corporatist society

- Anthropic asked for minimal limits on AI usage

- People view Anthropic's stand as heroic, while viewing OpenAI as villainous

- The false choice between Anthropic and OpenAI mirrors the false choice in American politics.

- People at OpenAI, Anthropic, and elsewhere used to view ethical deployment of AI as paramount, but those goalposts have shifted as financial and political incentives changed.

- Specifically, the ethics of AI have become conveniently synonymous with the current financial and political moment.

- The current political moment is fascist.

- Technology is broadly neutral and it is politics that primarily dictates how technology is actually used and deployed, and therefore its broad impacts.

- The internet was developed in the neoliberal era, which began with the election of Ronald Reagan and extended through the Obama presidency.

- The structure and dynamics of the internet over the last 30 years is more reflective of neoliberal politics than it is of anything inherent in the technology. Extreme privitization and the refusal to use public institutions to provision or regulate public goods.

- AI is being developed in a new political era, begun with the first Trump presidency, and taking more full shape under the second Trump presidency.

- We are likely to find that AI's trajectory is similarly dictated largely by politics rather than anything inherent to the technology.

- With this political era being fascist and explicitly neo-imperial/neo-colonial, I fear for the technology's impact on humanity.

- God help us.

It's funny how the false choice of American politics (Red vs Blue) also makes it into its consumerist corporatist life. That Anthropic's threadbare "limits" on government usage are seen as a heroic stand is a testament to just how far the goalposts on "ethical" deployment of AI have moved to the (fascist) right. As ever, politics precedes technology. We have Reagan's internet, we will have Trump's AI. God help us.

Looks nice!

I think this is spot on. It’s interesting how rhetoric about “liberty” seems to practically serve oligarchy. I suppose an alternative to bans and regulations is to genuinely pursue the elimination of deprivation, orient our collective capacities towards our collective well-being, and then let people do what they will. Anything short of that seems to be a rather false liberty (and a rather false democracy, while we’re at it).

Looks nice. Maybe share a bit about the implementation (windowing, networking, security, Mac APIs, …)?

Here here. Strategy and efficiency are overrated, desire (even obsession) underrated.

Yes, I think you're right. Seems to reveal a fundamentally extractive, rather than value-generative, economic model.

I always find it odd when media (and others) consider consumerism as somehow "helping" the economy. The economy is entirely about the collective activity of humans serving humans. Everything we make or do is really about prioritizing that activity over others. Why would it be advantageous to prioritize barely-distinguishable "new" devices over the myriad other things human labor and capital could be put to?

"Metaphorically, I think about any job given to Claude as having 3 dimensions. There's the breadth of the task (roughly how many lines of code it will touch), the depth of the task (the complexity, the layers of abstraction needed, the decision making involved, etc.), and the time spent working on it. Those three axes define a cube, and the size of the cube is how much entropy I'm shoving into the project."

That's an interesting conceptualization that tracks with my experience using CC. And they were able to get an impressive amount of work done:

"""

The specifics don't matter too much here, but for context, some of what I had it do:

- Research all the available on-device speech-to-text models with permissive licences

- Demo the transcription speed of each one on an android device attached to the PC

- Write a C wrapper for the best one (Moonshine) and build an embeddable dynamic library

- Build this for iOS, Android, Linux, and macOS, and integrate it with my app code using the FFI

- Build a Nim wrapper for the fdk-aac library

- Integrate it with miniaudio, so I can play AAC audio and pipe the audio into Moonshine

"""

Thinking of bunny huang’s open source hardware efforts, a 28nm nommu system seems like a good long term open source target. How much complexity in the system is the mmu, and so how much complexity could we cut out while still having the ability to run third-party untrusted code?

I’m sorry, I did not intend to accuse you of being part of the evangelical community. Your article only prompted the thought I shared.

On the technical point, I think I do disagree, but open to changing my mind. What would be better? I’m working on an async runtime currently, written in C, and I’m using several intrusive doubly linked lists because of their properties I mentioned.

That would be the “arbitrarily bounded” allocation. Arbitrary because now you have to make a decision about how many items you’re willing to maintain despite that number being logically determined by a sum over an unknown set of modules.

> The last time I used [a doubly linked list] was in the nineties. I know the Linux kernel uses them, but that design was also laid down in the nineties; if you were designing a kernel from scratch today, you would probably not do it that way.

This and sibling comments about the supposed uselessness or outdatedness of linked lists taught me something about the disconnect between many systems engineers resistant to Rust and many members of the (evangelical) Rust community.

To address the technical topic first: a doubly linked list is what you reach for when you’re looking for an intrusive data structure with fast append and fast delete. You need to iterate over it, but don’t need random access. Queues where you may need to remove elements. Think a list of completions that can be cancelled.

Why an intrusive data structure? Because you don’t want to, or can’t allocate. Because you want only a non-owning reference to the object, and you don’t want to have to manage any other lifetimes or dynamic (potentially unbounded, or arbitrarily bounded) allocations. Intrusive data structures are a great tool to minimize allocations, and to keep allocations (and ownership) logically separated across modules.

And now onto the cultural topic. That a member of the Rust community might not know this (which is of course totally fine) is what taught me something, which was maybe obvious to many: Rust brings safety to a C++ style language and audience. For the same reasons many dislike C++, many dislike Rust. For the same reasons many would opt for C over C++ when they have the choice (either across all their projects or for certain projects), many continue to opt for C over Rust.

Rust will not be taking over systems programming for the same reasons C++ did not. Well, by the numbers, it probably did (and Rust probably will too), so maybe better to say that it will not stamp out C for systems programming for the same reasons C++ did not. And it’s not just because of the stubbornness of C programmers.

Systems programming has 2 cultures and practice groups: the C camp, and the C++ camp. The C++ camp is a big tent. I’d argue it includes Rust and Swift (and maybe D). Zig belongs to the C camp.

Safety is important, and maybe Rust has the right model for it, but it is not a solution for the C camp. It likely did not intend to be, and again, by the numbers, replacing C++ is probably the more important task.

Certainly onto something but misses how much large organizations are actually controlled by small organizations operating in the “large complex system” environment. It is only individuals and small organizations that have agency at all. Large organizations and large complex systems are both emergent, one with hierarchical control, and one with distributed control. What has really changed is how unequal small organizations have become in their influence and power. The small cadres of people at the “top” (of organizations, media, government, tech, etc) control/influence more and more, not only at the expense of other small organizations (power is zero sum) but also at the expense of the decentralized mechanism, ie the large complex system becomes increasingly hierarchically/centrally controlled (vs distributed/decentralized control).

Yes being able to have compilers as libraries inline in the same code and same language. That feels like what all these call for. Which really is the Lisp core I suppose. But with static types and heterogenous backends. MLIR I think hoped (hopes?) to be something like this but while C++ may be pragmatic it’s not elegant.

Maybe totally off but would dependent types be needed here? The runtime value of one “language” dictates the code of another. So you have some runtime compilation. Seems like dependent types may be the language of jit-compiled code.

Anyways, heady thoughts spurred by a most pragmatic of libraries. Cloudflare wants to sell more schlock to the javascripters and we continue our descent into madness. Einsteins building AI connected SaaS refrigerators. And yet there is beauty still within.

I do find the pattern interesting and powerful.

But at the same time, something feels off about it (just conceptually, not trying to knock your money-making endeavor, godspeed). Some of the issues that all of these hit is:

- No printf debugging. Sometimes you want things to be eager so you can immediately see what's happening. If you print and what you see is <RPCResultTracingObject> that's not very helpful. But that's what you'll get when you're in a "tracing" context, i.e. you're treating the code as data at that point, so you just see the code as data. One way of getting around this is to make the tracing completely lazy, so no tracing context at all, but instead you just chain as you go, and something like `print(thing)` or `thing.execute()` actually then ships everything off. This seems like how much of Cap'n Web works except for the part where they embed the DSL, and then you're in a fundamentally different context.

- No "natural" control flow in the DSL/tracing context. You have to use special if/while/for/etc so that the object/context "sees" them. Though that's only the case if the control flow is data-dependent; if it's based on config values that's fine, as long as the context builder is aware.

- No side effects in the DSL/tracing context because that's not a real "running" context, it's only run once to build the AST and then never run again.

Of the various flavors of this I've seen, it's the ML usage I think that's pushed it the furthest out of necessity (for example, jax.jit https://docs.jax.dev/en/latest/_autosummary/jax.jit.html, note the "static*" arguments).

Is this all just necessary complexity? Or is it because we're missing something, not quite seeing it right?

This is cool.

There's an interesting parallel with ML compilation libraries (TensorFlow 1, JAX jit, PyTorch compile) where a tracing approach is taken to build up a graph of operations that are then essentially compiled (or otherwise lowered and executed by a specialized VM). We're often nowadays working in dynamic languages, so they become essentially the frontend to new DSLs, and instead of defining new syntax, we embed the AST construction into the scripting language.

For ML, we're delaying the execution of GPU/linalg kernels so that we can fuse them. For RPC, we're delaying the execution of network requests so that we can fuse them.

Of course, compiled languages themselves delay the execution of ops (add/mul/load/store/etc) so that we can fuse them, i.e. skip over the round-trip of the interpreter/VM loop.

The power of code as data in various guises.

Another angle on this is the importance of separating control plane (i.e. instructions) from data plane in distributed systems, which is any system where you can observe a "delay". When you zoom into a single CPU, it acknowledges its nature as a distributed system with memory far away by separating out the instruction pipeline and instruction cache from the data. In Cap'n Web, we've got the instructions as the RPC graph being built up.

I just thought these were some interesting patterns. I'm not sure I yet see all the way down to the bottom though. Feels like we go in circles, or rather, the stack is replicated (compiler built on interpreter built on compiler built on interpreter ...). In some respect this is the typical Lispy code is data, data is code, but I dunno, feels like there's something here to cut through...

I share the author's enthusiasm for coroutines. They're nice abstractions for all sorts of state-machine-like code and for concurrency (without parallelism).

> You could allocate a piece of memory for a coroutine stack; let the coroutines on it push and pop stack frames like ordinary function calls; and have a special ‘yield’ function that swaps out the stack pointer and switches over to executing on another stack. In fact, that’s not a bad way to add coroutines to a language that doesn’t already have them, because it doesn’t need the compiler to have any special knowledge of what’s going on. You could add coroutines to C in this way if you wanted to, and the approach would have several advantages over my preprocessor system.

In C minicoro is a nice library that provides just that: https://github.com/edubart/minicoro

In Zig there's zigcoro: https://github.com/rsepassi/zigcoro

Another source I found enlightening on coroutines is "Coroutines in Lua": https://www.lua.org/doc/jucs04.pdf

If we go a bit old-school on AI and reason in the "connectionist" framing:

Let's say neural memories are encoded in some high-dimensional vector space.

And so memory recall is an associative process that entails constructing a query vector and issuing it across the neural memory space.

And the brain is constantly learning, and that learning entails some changes in the structure of the high-dimensional memory space.

And let's say that re-encoding of a neural memory happens upon recall, and only upon recall.

Then it could be that all experience is in fact stored, but because of changes due to learning, those memories become inaccessible. The machinery constructing query vectors has updated its structure enough that its encoding of those query vectors is sufficiently dissimilar from the encoding of the stored memory vectors (which use the encoding from the last recall).

ref: http://www.paulgraham.com/submarine.html

Very cool! Love that it's about bringing joy to kids. Some of tech/AI feels soul-sucking at the moment. Can you tell us more about the technical challenges of making something like this run real-time locally? How do the methods compare to things like Apple's Animjois and TikTok's facial filters? And do you have plans to bring voice changes as well? Again, nice work!