porridgeraisin

> For Qwen, the model fixed a looping bug that may have been known apriori

It was included in the prompt itself. The authors say they used claude code for the experiment, and claude wrote the prompt for astra, which included details of the looping bug. Then astra seems to have done h(looping) - h(loop fixed) and then gotten the usual steering vector.

See my other comment in this thread

I am very aware, being indian myself, and have broad knowledge of the tech ecosystem here. Openai/anthropic in india is mostly sales. There is not yet research/engineering roles beyond that.

> but I won't be surprised if you could retire in India after working on $180k/yr for 2-3 years.

Irrelevant, but just for information, you could provided you're willing to live in a relatively rural area.

a) This is just really cool. I mean, one-shotting a game policy's weights is nuts. b) Plausibly, in its pretraining data various model weights were also included, which leads to why this is very important: interpretability

That an LLM was trained on various policy weights, along with labels of what that policy does (not labelled directly, but indirectly) and is then able to guess what game a policy is trained for purely from its weights, and one-shot generate weights for these games, means that there is some way to compress/recover/represent what an NN has learned purely from its weights, which is what interpretability is all about. Potentially, it could reveal interpretability tools much better than the ones we use today.

Edit: tweets from one of the coauthor providing more context:

https://x.com/a1zhang/status/2117328657161322810

https://x.com/a1zhang/status/2117330975122235442

This is overall correct. Just a nit: openai/anthropic don't have much of an engg presence in India. But yes for all of big tech.

AI can help with that too! Have it pwn the library's public endpoints to get access to their catalog.

> making it more of a fluff/marketing piece than an actually interesting technical blog in this area.

/genuine do you recommend any other blog?

> you need not only the pocket

Hmm.. I think it's my terminology issue. I just meant pocket as in informal word for "subspace", that is, generators can learn to generate good candidates within some subspace without needing to be good classifiers across the whole space.

But yeah as a whole that makes sense, thanks.

My reading of the hypothesis is this, interested in your thoughts:

> unless you can create a magical classifier

A generator could concentrate on pockets while remaining unable to judge most drugs. And LLM RLVR a) works well, and b) is black box optimisation and so is suited since it doesn't need system identification (which you can't do for complex biological systems). So the hypothesis is that, yes making a classifier is very very hard, but you could through some (lots?) initial interactive data, end up with our generator outputting with high probability drugs-that-end-up-passing-trials-with-probability-p. Whether these drugs thus generated are drugs we care about (i.e, whether `p` is good enough) and whether the whole pocket it generates in is large enough, and how jagged this behaviour is (i.e distribution of `p` across taxonomies of drugs), are the few questions we want answered.

You're right. I think I remember those numbers from an Air vs 3xxH comparison.

An example of this is that it expands the solution space to many things. At a minimum, many tedious things are no longer tedious. Selectively making use of this "tedium-canceler" in the right places is a skill and it can be a differentiator between people.

In usual developer type tasks, among my friend group what I hear is that panther lake laptops are competitive with M4 laptops. But, M4s do that with a 55Wh battery, whereas most panther lake laptops come with a 70Wh battery. So if you're discussing chip performance per watt then it's probably an edge to apple silicon.

This just seems like it's overfit to their RL environment, conditioned on the rather specific prompt they gave. Dressing it up this way feels like a marketing article for, probably, their sovereign AI product.