horseloverthin

Assuming no implicit phacking a 15 point jump on tbench scientific is a big deal.

I suspect we will see people being adamant that this model simply 'feels' better due to long horizon goal alignment improvements ( like this kind that would cause this jump)

formal algorithms for transformers provides perhaps the most succinct introduction to attention I can imagine https://arxiv.org/pdf/2207.09238