News for hackers. Stuff that matters.
Re:The Geometry of Categorical and Hierarchical Concepts in ... by Animats on Tuesday June 11, 2024 @03:48AM (#40643665)
Wow. This seems really important, because LLMs have been such black boxes.

Is this result useful only for basic concepts backed by huge numbers of cases in the training data, or is it more general than that?

Comments?

Re:The Geometry of Categorical and Hierarchical Concepts in ... by zmgsabst on Tuesday June 11, 2024 @08:16AM (#40645324)
This is generally true, about type theories:

A type theory corresponds to a complex diagram, as outlined in topos theory. (Note: complex as in CW-complexes.)

I think it’s fascinating LLMs ended up being a similar structure — but perhaps not entirely surprising. There have been similar results, eg a topological covering can generate an ML model.