Increase GPT4's context size by asking it to compress your prompts

Re:Increase GPT4's context size by asking it to compress you... by danShumway on Wednesday April 05, 2023 @09:29AM (#35453810)

If you look into this into more detail:

A) people have had a hard time reproducing it, and

B) more damning, the "compressed" version uses more tokens than the original (https://gist.github.com/VictorTaelin/d293328f75291b23e203e9d...)

> In practice, plain, well-designed summaries should be optimal to fit larger documents in the context.

> This concept has potential, though; building lookup tables seems to outperform long text summarization.

It's a clever idea, and I agree that lookup tables and external storage of memory is likely going to be important at some point, but I suspect that's going to come out of giving LLM more ability to externally reference "long-term" memory rather than compressing everything into immediate context.

[ Parent | on HN ]

Re:Increase GPT4's context size by asking it to compress you... by catchnear4321 on Wednesday April 05, 2023 @09:34AM (#35453894)

People are smashing the duplos together too hard.

Sessions are getting treated as more valuable than necessary.

This should look more like functions.

Needs lower prices and greater availability for that.

But until then… smashing duplos together.

[ Parent | on HN ]

Re:Increase GPT4's context size by asking it to compress you... by gfodor on Thursday April 06, 2023 @12:04AM (#35463995)

I spent some time on this, here is a better version: https://mobile.twitter.com/gfodor/status/1643415357615640577

[ Parent | on HN ]