you could probably ask it

I'd considered that -- in some cases it says it's absolutely confident, like with basic arithmetic, and in others it says:

> Without more information, I cannot estimate the accuracy or reliability of these claims. It is important to note that there may be conflicting or inaccurate information available online, and it is always a good idea to verify information from multiple sources before accepting it as fact.

which seems pretty reasonable.

You can ask it, and it may generate text with a score, but there is no particular reason that the score generated in the text would reflect any internal score it has.

I had a long requirements chat last night and did just that:

There are many self-hosted solutions that could potentially meet the requirements you have listed for your RSS aggregator and reader project. Here is a list of some popular options, ranked in order of their approximate percentage of compatibility with the features you have specified:

RSS Aggregator

| Percentage Score | URL |

| --- | --- |

| 90% | https://tt-rss.org/ |

| 85% | https://miniflux.app/ |

| 80% | https://newsboat.org/ |

| 75% | https://github.com/lobsters/lobsters |

| 70% | https://github.com/noahcoad/feeds |

| 65% | https://github.com/ssut/py-googlenews |

| 60% | https://git.gnome.org/browse/feedreader/ |

RSS Reader

| Percentage Score | URL |

| --- | --- |

| 90% | https://tt-rss.org/ |

| 85% | https://miniflux.app/ |

| 80% | https://newsboat.org/ |

| 75% | https://github.com/lobsters/lobsters |

| 70% | https://github.com/noahcoad/feeds |

| 65% | https://github.com/ssut/py-googlenews |

| 60% | https://git.gnome.org/browse/feedreader/ |

Please note that these percentages are approximate and are intended only as a rough guide. You may want to consider other factors such as the user interface, performance, security, and integration with other tools when choosing a solution for your project.

These scores would mean nothing! At best it would score "how well does my answer fit to my model of the English language". At no point can the current architecture accurately grade "how correct is this answer in the context of the subject domain".

The GPT-3 playground lets you view the probabilities of generated tokens. That's the closest thing to a confidence score you can get from a transformer model. But like the other person said, this isn't very meaningful.