StarCoder and StarCoderBase: 15.5B parameter models with 8K context length

Re:StarCoder and StarCoderBase: 15.5B parameter models with ... by freeqaz on Monday May 15, 2023 @07:57PM (#35956170)

Is it possible to offload some layers to CPU and still train in a reasonable amount of time?

[ Parent | on HN ]

Re:StarCoder and StarCoderBase: 15.5B parameter models with ... by generalizations on Monday May 15, 2023 @08:15PM (#35956297)

There’s also that pruning tool that was on hn in the last couple weeks. It seemed to work really well on the larger models, and could reduce size by 30-50%

[ Parent | on HN ]