TopoNets: High performing vision and language models with brain-like topography

Re:TopoNets: High performing vision and language models with... by harles on Thursday January 30, 2025 @11:21PM (#42884665)

That could explain compute efficiency, but has nothing to do with the parameter efficiency pointed at in the paper.

[ Parent | on HN ]

Re:TopoNets: High performing vision and language models with... by vlovich123 on Friday January 31, 2025 @10:57AM (#42888824)

Haven’t read the paper but my guess around that is that the same reason sparse attention networks (where they 0 out many weights) just have the sparse tensors be larger.

[ Parent | on HN ]

Re:TopoNets: High performing vision and language models with... by mayukhdeb on Saturday February 01, 2025 @08:57AM (#42898307)

In this paper, we don't zero out the weights. We remove them.

[ Parent | on HN ]

Re:TopoNets: High performing vision and language models with... by vlovich123 on Sunday February 02, 2025 @03:40PM (#42911573)

Thanks for the correction! Can it be retrofitted into existing models through distillation or do you have to train the model from scratch?

[ Parent | on HN ]