News for hackers. Stuff that matters.
Re:The Secret Sauce behind 100K context window in LLMs: all ... by stoniejohnson on Sunday June 18, 2023 @12:25AM (#36377311)
So since a model is fine tuned via RLHF my point doesn't stand?

Genuine question; it would be interesting if some other mechanism was at play here.

Re:The Secret Sauce behind 100K context window in LLMs: all ... by jumpCastle on Sunday June 18, 2023 @12:42AM (#36377382)
For an answer I would expect it to get the same reward for both question orderings. So naively I would expect it to not be affected by the ordering.