Re:GPT-J-6B: 6B JAX-Based Transformer
by radarsat1 on Wednesday June 09, 2021 @06:35AM
(#27446355)
I suspect this is at least partly because of they way they are used (and maybe trained?)
Always, "given a prompt, keep talking." No instructions to go anywhere, so it's no surprise that they do not.
I think, "start with this idea, end with this one" should give much more interesting results. Telling it to start with a premise and come up with the filler needed to draw some conclusion. It would give it more of a target for making long-distance connections.
Otherwise you just get this open-loop blabbering, which I agree seems really useless. With a more "directed" model I can see this having actual applications, like with story writing or interactive video games. But as it stands this seems totally uninteresting from an applications point of view.
Always, "given a prompt, keep talking." No instructions to go anywhere, so it's no surprise that they do not.
I think, "start with this idea, end with this one" should give much more interesting results. Telling it to start with a premise and come up with the filler needed to draw some conclusion. It would give it more of a target for making long-distance connections.
Otherwise you just get this open-loop blabbering, which I agree seems really useless. With a more "directed" model I can see this having actual applications, like with story writing or interactive video games. But as it stands this seems totally uninteresting from an applications point of view.