gmork13

So it can load twice as large models somehow?

This is from today apr 5 saying the mmap change loads twice as big models with x100 speed up - is this not a blatant lie?

Wasn’t it discovered last week that loading larger models was an error in measurement and the speed up was from keeping things in memory after the first loading?

Please do correct me if I’m wrong.

Was that implementation confirmed to work or was it a bug?

I saw people saying it only loads the weights necessary, but all the weights are necessary (unless the architecture is heavily modified). Does it stream or load only the currently necessary layers in a way?