Unsloth creators fix universal error with gradient accumulation

Posted by ZQ-Dev8 on Wednesday October 16, 2024 @06:30PM
from the unsloth.ai dept.

( 2 comments | 2 points | on HN )

Re:Unsloth creators fix universal error with gradient accumu... by gnabgib on Wednesday October 16, 2024 @07:24PM (#41864915)

Article title: Bugs in LLM Training - Gradient Accumulation Fix

[ Parent | on HN ]

Re:Unsloth creators fix universal error with gradient accumu... by ZQ-Dev8 on Wednesday October 16, 2024 @06:31PM (#41864511)

seems like something pytorch maintainers would want to know about and fix asap...

[ Parent | on HN ]