I asked GPT-3 for the question to “42”

Re:I asked GPT-3 for the question to “42” by cblconfederate on Wednesday November 25, 2020 @04:05AM (#25207277)

Why do people expect that gpt has evolved to a semantic engine, since nothing of the sort was engineered into it? It's comparable to reflex neural circuits , or to central pattern generators, but not to (yet unknown) cognitive centers

[ Parent | on HN ]

Re:I asked GPT-3 for the question to “42” by tiborsaas on Wednesday November 25, 2020 @07:15AM (#25208397)

The transformer model is in fact engineered to discover semantics.

https://en.wikipedia.org/wiki/Transformer_(machine_learning_...

Given a large enough training dataset it shows that intelligent behavior arises, which is pretty remarkable. It can also be scaled further.

Read the original paper on the architecture: https://arxiv.org/abs/1706.03762

[ Parent | on HN ]

Re:I asked GPT-3 for the question to “42” by cblconfederate on Wednesday November 25, 2020 @07:26AM (#25208484)

I'm familiar. It's engineered to attend , attend , attend and fill in blanks. Nowhere is there any consideration for semantics

[ Parent | on HN ]

Re:I asked GPT-3 for the question to “42” by tiborsaas on Wednesday November 25, 2020 @07:45AM (#25208624)

Weird, when I spent lots of time with Philosopher AI all I've seen is semantics in action. How else do you expand on a topic based on a simple question and keep the context? How can it then compress text to summarize it well? How can it obtain high level answers for complex questions?

[ Parent | on HN ]

Re:I asked GPT-3 for the question to “42” by cblconfederate on Wednesday November 25, 2020 @07:55AM (#25208702)

To answer these questions one would need to begin with rigorous definitions of 'semantics', 'context', 'memory', 'complex' etc. What we know is that attention models are heavily trained associative memories. Anecdotal perceptions are deceiving, as humans we are often prone to see human/animal-like behavior where it doesnt exist (e.g. cgi/games).

[ Parent | on HN ]

Re:I asked GPT-3 for the question to “42” by TheUndead96 on Wednesday November 25, 2020 @06:33AM (#25208165)

I think this is actually the interesting part. It appears to understand some causality in spite of never being trained this. GPT-3 seems to be able to do arithmetic with small numbers, but begins to fail with larger equations. This seems congruous to how humans may internalise arithmetic. As far as we know, we do not have an ALU in the brain.

[ Parent | on HN ]

Re:I asked GPT-3 for the question to “42” by cblconfederate on Wednesday November 25, 2020 @06:41AM (#25208209)

We are also known for seeing patterns where they don't exist. If someone is going to make claims about the neural network's reasoning they should corroborate it with statistical analysis of its performance

[ Parent | on HN ]