News for hackers. Stuff that matters.
Re:A global workspace in language models by echelon on Monday July 06, 2026 @05:20PM (#48810668)
> I get the feeling a lot more research is going to come out in the area of exploring exactly what portions of a model's weights do what.

Too bad the frontier models are closed weights.

Maybe the research community and whole rest of the world will build on open and all the advances will happen in open ecosystems instead.

Re:A global workspace in language models by ayewo on Monday July 06, 2026 @05:45PM (#48810896)
A Google DeepMind researcher (Neel Nanda) was able to replicate their claims on an open weight model (Qwen 3.6 27B):

> We have replicated the core claims on Qwen 3.6 27B, and also share preliminary evidence of extending this work by finding abstract "interpretative meta-tokens", like Chinese characters for "what does this mean" that seem to activate and play a causal role on processing ambiguous sentences

See p33 of [1]

Anthropic also released companion code to go with their paper in [2] which also used Qwen. They state that their code should be broadly adaptable to other open weight models with HuggingFace decoders.

[1]: https://www-cdn.anthropic.com/files/4zrzovbb/website/cc4be24...

[2]: https://github.com/anthropics/jacobian-lens

Re:A global workspace in language models by dnhkng on Tuesday July 07, 2026 @09:02AM (#48817177)
Too bad they also don't give anything back to individual researchers. Oh well, wasn't expecting much.