leumassuehtam
Re:Ts_zip: Text Compression Using Large Language Models
on Wednesday August 16, 2023 @04:20PM (#37153167)
Interesting application for the RWKV family of models.
Re:How the RWKV language model works
on Tuesday July 04, 2023 @09:11AM (#36586198)
I think RWKV team should make a statement by training a really large model. Their 14B finetunes are fine but far from impressive. If they believe that this is the future, they should have at it. They probably have funding to do so (I think?).
Anyway, exciting to see what comes next.
Replit Unveils LLM with 2.7B Parameters and 20 Languages
Bert and XLNet: 85% Neurons Redundant, 92% Removable for Downstream Tasks [pdf]
Codeium AI Chat Is Here
Re:The RWKV language model: An RNN with the advantages of a transformer
on Thursday March 30, 2023 @07:00AM (#35370783)
You can currently play around with the 7B[0] and 14B[1] parameters models hosted in Hugging Face Spaces.
Re:Faster Than Light in Our Model of Physics: Some Preliminary Thoughts
on Friday October 02, 2020 @01:28PM (#24664875)
The concept of updating rules underlies this model. Each rule determine the evolution of the hypergraph which is space itself, much like a cellular automata. It seems that some rules have the property that as "times goes by" the hypergraph converges to a manifold with dimension d. One can estimate the dimension of such hypergraph using, for example, the growth of the volume of balls[0].
It also seems that this concept is very generic, and many properties of the model do not depend on the dimensionality neither on the manifold or the rule itself.
Because this concept is so generic, one hopes that there is a rule that the hypergraph converges to d=3 as in our world. This would also raise questions on why in the very early universe there was a "inflationary" phase.
[0] https://www.wolframphysics.org/technical-introduction/limiti...