cacio-e-pepe

Thoughts on the bitter lesson? Eg what if models just keep getting better?

Could you expand? Curious.

relevant: https://mathstodon.xyz/@tao/117219101339291693

Nice, looking forward to the report.

And thanks, huge pasta fan :)

> So we did mechanistic studies on small models, Gemma 4 particularly, and found the hidden state for different layers carry meaningful self-awareness signal for various situations.

Neat! Just to make sure I understand - you trained your probe layer to take this hidden state and predict p(wrong)?

Curious to learn more. Any more info on your approach (esp the mechanistic study)?

Honestly, stellar performance by the model at the capability being measured.

Okay so the diffusion model generates image assets, and you iterate on those, and then another model turns your favorites into code, is that right?

Looks interesting. I'm a bit too busy to read the docs, but I'm curious - how does it work?

What are the core knobs for a RAG pipeline? See any interesting patterns? Eg in practice, what did the agents tend to tune? Did domain matter?