Re:Knowledge Is a Region in Weight Space for Fine-Tuned Lang...
by barking_biscuit on Thursday February 23, 2023 @04:12PM
(#34916532)
>Because of randomization in initialization and training
Randomization in initialization seems like a pragmatic thing to do from a 'make the math work' perspective, but a really counter-intuitive thing to do when comparing the training process of our own wetware substrate. I know it's not a fair comparison, but just an interesting thought to me.
Randomization in initialization seems like a pragmatic thing to do from a 'make the math work' perspective, but a really counter-intuitive thing to do when comparing the training process of our own wetware substrate. I know it's not a fair comparison, but just an interesting thought to me.