A look at Apple’s new Transformer-powered predictive text model

Re:A look at Apple’s new Transformer-powered predictive text... by codeflo on Sunday September 17, 2023 @01:00AM (#37541671)

And interesting test. I didn’t see a mention of the temperature setting used. Temperature controls the probability to pick a token that isn’t the top prediction, which leads to more creative/less robotic results.

For actual input prediction, you probably want the temperature to be zero. But even a model as good as GPT-3 becomes very boring and repetitive with those settings.

[ Parent | on HN ]

Re:A look at Apple’s new Transformer-powered predictive text... by jackcook on Sunday September 17, 2023 @08:02AM (#37544132)

I used greedy sampling (temperature 0) for all of them. Since I didn't have access to logits/probabilities for Apple's model, I wasn't able to do anything else in a way that would be fair.

[ Parent | on HN ]

Re:A look at Apple’s new Transformer-powered predictive text... by smpanaro on Sunday September 17, 2023 @10:24AM (#37545360)

There is an input for temperature in the CPU model. If you can find and hook the call to predict you can probably see what is being passed.

Interestingly the Neural Engine version of the model does not take a temperature input, but it does output the raw embeddings in addition to the logits.

[ Parent | on HN ]