News for hackers. Stuff that matters.

apophis-ren

Re:Q-learning is not yet scalable on Sunday June 15, 2025 @02:09PM (#44283935)
It's mentioned in the article. But for really, really long-horizon tasks, it might be reasonable that you don't want to have a small discount factor.

For example, if you have really sparse rewards in a long-horizon task (say, a reward appears 1000 timesteps after the action), then a discount factor of even 0.99 won't help to capture that: 0.99 ^ 1000 ≈ 4e^-5.

Essentially, if your discount factor is too small for an environment, it will be near impossible to learn certain credit assignments.

Re:Tensor Product Attention Is All You Need on Sunday February 02, 2025 @10:45PM (#42914705)
Flash attention is an implementation trick; you can implement MHA/GQA, for example, with flash attention.