Re:Scaling Transformer to 1M tokens and beyond with RMT
by ca_tech on Monday April 24, 2023 @10:50AM
(#35688355)
You can work around this by chunking the data using a sliding window. Take the first X amount of sentences that fit into the token length and summarize. Now slide your window of sentences so that you still overlap just a bit with your previous selection. Summarize that information. Continue until you have a bunch of summaries for your text. You can then pass that back into the model for a more concise summary of the summaries.
[ Parent
| on HN ]