News for hackers. Stuff that matters.
Re:Input to GPT Matters a Lot by BiasRegularizer on Monday April 03, 2023 @11:35AM (#35425541)
It's not impossible to see that LLMs can make finite step "reasoning" between input and output, as each block of the transformer can model probabilistic causality. Transformers are sometimes considered a fully connected graph neural network, which can be used for modeling causal graphs. One additional supporting evidence on the finite step reasoning hypothesis is that "train of thought" improves the performance of LLMs[1], meaning that letting the model explain itself explicitly reduces the amount of implicit "reasoning" steps that need to happen within the model.

Additionally, while we don't know how us humans functionally reason, it's believed that the predictive nature of our brain is central to our reasoning abilities. In some way, it's similar to the autoregressive of LLMs.

For a collection of emergent properties in LLMs, I recommend this paper [2]

[1]https://arxiv.org/abs/2201.11903.pdf [2]https://arxiv.org/pdf/2206.07682.pdf

Re:Input to GPT Matters a Lot by tomrod on Monday April 03, 2023 @11:48AM (#35425815)
Thanks for the references, will review.