Huawei releases an open weight model trained on Huawei Ascend GPUs

Re:Huawei releases an open weight model trained on Huawei As... by coolspot on Wednesday July 02, 2025 @02:23PM (#44447136)

Possible and has been done, but super-slow and inefficient resulting in long training times for small models. To keep compute occupied you need to pass gradients very fast.

[ Parent | on HN ]

Re:Huawei releases an open weight model trained on Huawei As... by reactordev on Wednesday July 02, 2025 @04:13PM (#44448289)

Yes but could you break it up into chunks of sets of gradients to compute? I know that compute needs the full chunk to compute a set. Again, things I’m exploring but ultimately no different than just having the full dataset on disk and just scaling out compute nodes in ro mode.

[ Parent | on HN ]

Re:Huawei releases an open weight model trained on Huawei As... by pk-protect-ai on Wednesday July 02, 2025 @02:39PM (#44447310)

Do you mean this one?

https://blog.lambdaclass.com/introducing-demo-decoupled-mome...

[ Parent | on HN ]

Re:Huawei releases an open weight model trained on Huawei As... by reactordev on Wednesday July 02, 2025 @04:15PM (#44448313)

This is what piqued my interest in the first place

[ Parent | on HN ]