My solution to this (not yet integrated) is to use a faster / cheaper model to process all the text first and see if it's actually relevant or not, before running the summarization / task prompt with the main model.
I realized that this is essentially a semantic search engine for text. Using the same principle you can feed a very large text file in, ask it a question, and it finds the page that answers that question.
This would be useful as a layer "beneath" the internet research agent, that it could use to sort through all the noise and answer the question.
https://gist.github.com/avelican/58958a2cf2b7e9f9f555ab94549...
It gives a lot of false positives, I'm not sure if this is a limitation of GPT-3 (have not tried GPT-4 for this yet, a bit too expensive for searching books) or a limitation of my implementation.
It's probably slower and more expensive than just using a vector db, I haven't tried those yet.
I think this is due to the main loop being a GPT feedback loop. Each loop it has a small chance of something going wrong (or a large chance, depending on the query), so as it loops repeatedly, the chance of failure approaches 100%.
My idea was to replace the core loop, instead of a GPT feedback loop just make it a few lines of Python.
Now the thing actually does what it says it's going to do, "thinking lag" is eliminated, and API usage is reduced 80%.
I turned (parts of) Auto-GPT into a tiny Python library, specialized for internet research.
GPT-3 and GPT-4 are able to use this library to write Python programs that do useful work.
This way they can "crystallize" their plans in code, to ensure that they will run.
Here is the interface:
See the comments below the gist for the GPT-3 and 4 versions of main.py (20-30 lines for an internet research agent!).https://gist.github.com/avelican/2d4e718954593e3df9e0e5ee675...
Note: It's currently optimized for my main use-case, which is internet research. So it's not an Auto-GPT in any sense. But it does one thing, and does it fairly well.