Yeah the [Info] button says so. Quoting... Info About

Ask Paul Graham is a for-fun side project powered by OpenAI's GPT-3 and Paul Graham's essays. OpenAI capped costs at $360 per month which is currently being covered by the project maintainer so usage may get throttled depending on demand. Questions

The questions feature answers user-provided questions using Graham's essays as training data. Note that these are answers from GPT-3 and do not necessarily reflect Paul Graham's opinions. Summaries

The summaries feature provides GPT-3-generated summaries of Graham's essays and may not necessarily reflect his summary of the given essay. Not all essays have been included due to length constraints on GPT-3.

Yep! And I'd love ways to tweak/improve the responses.

Are you using their semantic search API for pre-filtering response candidates?

I'm following along their question answering guide: https://beta.openai.com/docs/guides/answers

I haven‘t been active and don‘t know if they have changed this. But when I was using GPT-3, their semantic search endpoint didn‘t actually use GPT-3 for ranking. It used some other, „cheaper“ algo as a pre-filter (tf-idf, I believe). And only the those results that this „cheaper“ algo returned on top made it into the evaluation by the actual GPT-3 model you had chosen.

(There was a setting that controlled how much of your data was actually ranked by GPT-3 - top_k or something. But to chose anything even approaching a sizable percentage of a large corpus such as the one you are dealing with would be prohibitively expensive. You‘d be looking at several Dollars per search.)

Anyway. This is also why switching the model from Ada to Curie to Davinci on the semantic search endpoint has (or had?) very little effect. It still often missed highly relevant snippets from the source material.

When they came out with their „answers endpoint“, it really was just a semantic search piped into a completion prompt.

I actually found it better to keep my own implementation of this pipeline, because then you at least get to control how the results of the semantic search are prompted to the generation model.