> I think Astra was trained on neural weights data as an attempt by OpenAI to get to recursive self improvement.

This might be the case, a few weeks ago, I wanted to compare a small neural network trained with AdamW vs a handcrafted set of weights on a random task. Astra was able to handcraft a set of weights that perfectly fit the small neural network to the task. AdamW was unable to tune the weights to get the same result.

I believe we've known humans can handcraft weights much more efficiently than standard gradient descent methods, we just don't have enough memory or time to scale ourselves, so it makes sense to train a neural network to do it on our behalf. I am excited to see how far we can push this method.

Mentioned chat: https://chatgpt.com/share/6ae8f024-34ec-83e8-9572-1300f52c80...

Research on using LLMs and diffusion models to directly predict weights have been going on for a bit now for example https://arxiv.org/pdf/2402.18153

It’s a good reminder of one of the most unique features of alphago - it was like a seven layer or so neural net that could predict out a hundred moves. I’m not sure we took this kind of compression super seriously or at least I would say perhaps it remains under explored. You could think of what you did as sort of the same thing as skipping the Monte Carlo search phase for a go engine — a process compression.

It looks like Astra picked the "random task", and the task it picked was 8-bit parity, which is easy to design a circuit for that computes the answer perfectly, but sort of maximally difficult to fit with any kind of gradient descent. It's a great choice for what you asked, which was to find a case where AdamW doesn't come close to the theoretical optimum fit. But I'm less convinced that it tells us much about Astra's capabilities at handcrafting weights in general.

8-bit parity is a^b^c^d^e^f^g^h, which you can compute with minimal depth as ((a^b)^(c^d))^((e^f)^(g^h)). You can compute x^y as x&~y | y&~x. With ReLU activation, i | j is ReLU(i + j): weights of 1, bias of 0. And i & ~j is ReLU(i - j): weights of 1 and -1, still bias of 0. So a two-input XOR is three neurons in two layers, and the tree of 7 binary XORs is 21 neurons in 6 hidden layers. I don't know why Astra needed 433 parameters. Only 42 nonzero parameters are needed. Maybe the other 391 parameters are zeros that the Transformer model demands to be present.

With an RNN you can compute the parity of a string of any length with only three neurons.

Actually, I handed that to Claude Opus 5.5 to check, and it sensibly pointed out that x&~y is mutually exclusive with y&~x, so XOR doesn't even need the third neuron, except in the last layer; you can just fold the addition into the weights of the next layer, which means you only need 4 layers and 15 neurons. But about half of those neurons need 4 nonzero input weights instead of 2, so you still need 42 nonzero weights. It's a good thing you don't need those OR neurons, too, because my definition is wrong: ReLU(1+1) = 2.

Also it explained how to design a 28-parameter net that I don't quite understand, and also a single-hidden-layer network with 8 ReLUs, which I also don't understand, but it needs more parameters: $\text{parity}=s+\sum_{k=1}^{7}2(-1)^k\,\mathrm{ReLU}(s-k)$ where I think $s=\sum_i x_i$.

Somewhat more surprisingly, it also said it didn't have any trouble fitting a 641-parameter network to this problem with AdamW.

Here is the code Opus 5.5 is claiming fits a 641-parameter MLP to parity of 8 inputs:

  # Fits small MLPs to 8-bit parity (the full 256-row truth table) with
  # full-batch AdamW, written out in numpy (no PyTorch, so it won't match a
  # PyTorch run bit for bit), and reports for each shape how many of 20 seeds
  # reach 100% accuracy and the median accuracy. Needs Python 3.9+ and numpy.
  import itertools

  import numpy as np

  X = np.array(list(itertools.product([0, 1], repeat=8)), float)  # all 256 inputs
  y = X.sum(1) % 2


  def nparams(widths: list[int]) -> int:
      """Parameter count of a dense MLP with these layer widths, biases included."""
      return sum(a * b + b for a, b in zip(widths, widths[1:]))


  def train(
      widths: list[int],
      seed: int,
      steps: int = 4000,
      lr: float = 1e-2,
      wd: float = 1e-4,
  ) -> float:
      """Full-batch AdamW on all 256 inputs; returns the final training accuracy."""
      rng = np.random.default_rng(seed)
      P = []    # parameters: W0, b0, W1, b1, ...
      for a, b in zip(widths, widths[1:]):
          bound = 1 / np.sqrt(a)    # same as PyTorch nn.Linear's default init
          P += [rng.uniform(-bound, bound, (a, b)), rng.uniform(-bound, bound, b)]

      m = [np.zeros_like(p) for p in P]    # Adam first moments
      v = [np.zeros_like(p) for p in P]    # Adam second moments
      nlayers = len(widths) - 1
      for step in range(1, steps + 1):
          hs = [X]    # hs[i] is the input to layer i
          zs = []     # pre-activations
          h = X
          for i in range(nlayers):
              z = h @ P[2 * i] + P[2 * i + 1]
              zs.append(z)
              h = np.maximum(z, 0) if i < nlayers - 1 else z    # ReLU; linear output
              hs.append(h)

          logit = h[:, 0]
          prob = 0.5 * (1 + np.tanh(logit / 2))    # sigmoid without overflow
          g = ((prob - y) / len(y))[:, None]    # d(mean cross-entropy)/d(logit)
          G = [None] * len(P)    # gradients, parallel to P
          for i in reversed(range(nlayers)):
              G[2 * i] = hs[i].T @ g
              G[2 * i + 1] = g.sum(0)
              if i > 0:
                  g = (g @ P[2 * i].T) * (zs[i - 1] > 0)

          # AdamW, using PyTorch's default values for betas (0.9, 0.999) and eps (1e-8)
          for k in range(len(P)):
              P[k] -= lr * wd * P[k]    # decoupled weight decay
              m[k] = 0.9 * m[k] + 0.1 * G[k]
              v[k] = 0.999 * v[k] + 0.001 * G[k] ** 2
              P[k] -= lr * (m[k] / (1 - 0.9 ** step)) / (
                  np.sqrt(v[k] / (1 - 0.999 ** step)) + 1e-8)

      # accuracy of the last forward pass, i.e. before the final update
      return ((logit > 0) == (y > 0.5)).mean()


  for w in [[8, 1, 2, 2, 2, 1], [8, 8, 4, 2, 1], [8, 8, 1], [8, 16, 1], [8, 64, 1]]:
      accs = [train(w, seed) for seed in range(20)]
      print(w, nparams(w), "params; seeds at 100%:", sum(a == 1 for a in accs),
            "/ 20; median acc", np.median(accs))

I might not know enough about ANNs to spot if it's cheating on some way, but it looks legit. It does indeed output that the [8, 64, 1] configuration gets 100% accuracy on all 20 seeds.

At least this slice

          h = X
          for i in range(nlayers):
              z = h @ P[2 * i] + P[2 * i + 1]
              # ...
              h = np.maximum(z, 0) if i < nlayers - 1 else z    # ReLU; linear output
              # ...

          logit = h[:, 0]

      # ...
      # accuracy of the last forward pass, i.e. before the final update
      return ((logit > 0) == (y > 0.5)).mean()

looks like a perfectly orthodox evaluation of a multilayer perceptron, with weights and biases in P, and that's what it's evaluating the accuracy on. I dumped out P for one of the [8, 64, 1] networks, and it does indeed look like about 650 parameters. I was going to post it here, but printed out as text it's like three pages long, so I won't.

It might also be a side effect of training models to help out with mechanistic interpretability, which I guess would help out a lot with recursive self-improvement.

I had the same thought, but why would they do this training on a model they release publicly? I assume they’d be doing that kind of RL on an internal fork only?

All NN behaviour is emergent strictly speaking; by definition. I think you mean a phase shift of a kind; a qualitative, unpredictable leap in capability.

Same thoughts. I doubt if this work can be reproduced with other frontier models. The title seems way too specific for Astra.

This is a great point / theory.

It would be really funny if it turns out every single openAI advancement just comes from cheating by learning the answers in advance.

Eg when they were caught stealing people’s maths research and then saying their model did it

They actually did math no one could.

They weren't caught stealing other people's maths research.

I think a lot of past examples of "emergent capabilities" have at least partly turned out to be that more stuff was included in the training data that (intentionally or not) covered the capability.

This is actually not new, I believe GPT 5.2 already had remarkable similar capabilities. I wrote a very elaborate private benchmark involving hand-authoring highly optimised transformer circuitry, and I was shocked that GPT 5.2 could do it. Makes sense that it was specifically trained on building and debugging transformers.

I am not smart enough to know the answer to this, so I am wondering if someone can help me with this question: if Astra is able to decipher structures and patterns within model weights, would an LLM be able to directly modify its own weights on-the-fly to achieve recursive self-improvement using a reasoning loop?