News for hackers. Stuff that matters.

A comment on T0* – Series of encoder-decoder models trained on a large set of different tasks. Parent comment

Re:T0* – Series of encoder-decoder models trained on a large... by hervature on Monday October 18, 2021 @11:37AM (#28906584)
Running the model multiple times doesn't reinforce the model. In general, you should not anthropomorphize algorithms as human cognition does not give any bearing on how algorithms work.
[ Parent | on HN ]
Re:T0* – Series of encoder-decoder models trained on a large... by Scene_Cast2 on Monday October 18, 2021 @11:52AM (#28906811)
It can. Check out "zero shot learning" -> both sentences would be part of a single "evaluation", and the first sentence would prime for the output of the second. (You basically combine multiple "evaluations" into one, and context is held in tensors / blobs)

https://towardsdatascience.com/zero-and-few-shot-learning-c0...

[ Parent | on HN ]
Re:T0* – Series of encoder-decoder models trained on a large... by hervature on Monday October 18, 2021 @05:26PM (#28911986)
Sure, but I feel like we're talking about different things. I consider "context held in tensors" as part of the model. That is, if you zero out these registers, then the model evolves in a deterministic way every time. In this case, when you perform a query, I assume those tensors are always initialized before your query.
[ Parent | on HN ]