But how would it know which weights to update? It has 3.5 and 3.8 weights memorized in its own weights and can somehow figure out which ones are responsible for different skills just in its own reasoning trace? (Since it didn’t use any external tools)?

That’s pretty crazy because, as I said in another comment, I just asked ChatGPT how many r’s are in blackberry and it said 3. I know it’s not Astra and the app doesn’t tell me which model it is, but I’m still very confused about how a model can edit neural net weights but not count letters in a reasoning trace.

Edit: Oh I misread, it didn’t use tool calls for the first part but it did have a custom harness for the part where it edited a models weights. Still impressive but not existentially confusing anymore.