News for hackers. Stuff that matters.

A comment on T0* – Series of encoder-decoder models trained on a large set of different tasks. Parent comment

Re:T0* – Series of encoder-decoder models trained on a large... by Mordisquitos on Monday October 18, 2021 @11:32AM (#28906494)
To be fair, if a real human were to answer the question "How many hydrogen atoms are in a water molecule?" time and time again, it would be very easy for them to accidentally reply "two" when asked the same question about oxygen.

The real question is, after the model mistakenly replied "two" to your question, did it also internally trigger the neurons for "Wait a minute..." while inhibiting output?

[ Parent | on HN ]
Re:T0* – Series of encoder-decoder models trained on a large... by hervature on Monday October 18, 2021 @11:37AM (#28906584)
Running the model multiple times doesn't reinforce the model. In general, you should not anthropomorphize algorithms as human cognition does not give any bearing on how algorithms work.
[ Parent | on HN ]
Re:T0* – Series of encoder-decoder models trained on a large... by Scene_Cast2 on Monday October 18, 2021 @11:52AM (#28906811)
It can. Check out "zero shot learning" -> both sentences would be part of a single "evaluation", and the first sentence would prime for the output of the second. (You basically combine multiple "evaluations" into one, and context is held in tensors / blobs)

https://towardsdatascience.com/zero-and-few-shot-learning-c0...

[ Parent | on HN ]
Re:T0* – Series of encoder-decoder models trained on a large... by hervature on Monday October 18, 2021 @05:26PM (#28911986)
Sure, but I feel like we're talking about different things. I consider "context held in tensors" as part of the model. That is, if you zero out these registers, then the model evolves in a deterministic way every time. In this case, when you perform a query, I assume those tensors are always initialized before your query.
[ Parent | on HN ]
Re:T0* – Series of encoder-decoder models trained on a large... by drran on Monday October 18, 2021 @03:24PM (#28910307)
How are in a water molecule?

> two hydrogen atoms

water molecule?

> isotopes of hydrogen

How water molecule?

> is arranged in a tetrahedral structure

[ Parent | on HN ]