News for hackers. Stuff that matters.
Re:Exploring LoRA – Part 1: The Idea Behind Parameter Effici... by threepi on Tuesday December 24, 2024 @11:20PM (#42506799)
Author here. Happy to see this posted here. This is actually a series of blog posts:

1. Exploring LoRA — Part 1: The Idea Behind Parameter Efficient Fine-Tuning and LoRA: https://medium.com/inspiredbrilliance/exploring-lora-part-1-...

2. Exploring LoRA - Part 2: Analyzing LoRA through its Implementation on an MLP: https://medium.com/inspiredbrilliance/exploring-lora-part-2-...

3. Intrinsic Dimension Part 1: How Learning in Large Models Is Driven by a Few Parameters and Its Impact on Fine-Tuning https://medium.com/inspiredbrilliance/intrinsic-dimension-pa...

4. Intrinsic Dimension Part 2: Measuring the True Complexity of a Model via Random Subspace Training https://medium.com/inspiredbrilliance/intrinsic-dimension-pa...

Hope you enjoy reading the other posts too. Merry Christmas and Happy Holidays!

Re:Exploring LoRA – Part 1: The Idea Behind Parameter Effici... by 3abiton on Wednesday December 25, 2024 @06:38AM (#42508312)
Thanks for sharing. This got me thinking, why is medium so used for such technical articles? Especially that lots of articles get blasted behind a paywall for me recently.
Re:Exploring LoRA – Part 1: The Idea Behind Parameter Effici... by ivanmontillam on Thursday December 26, 2024 @05:22AM (#42514293)
Making it less accessible, right? I was thinking exactly the same.
Re:Exploring LoRA – Part 1: The Idea Behind Parameter Effici... by anshumankmr on Thursday December 26, 2024 @05:59AM (#42514465)
short answer: to make money
Re:Exploring LoRA – Part 1: The Idea Behind Parameter Effici... by gautambt on Tuesday December 24, 2024 @11:55PM (#42506934)
Re:Exploring LoRA – Part 1: The Idea Behind Parameter Effici... by khazhoux on Wednesday December 25, 2024 @04:18PM (#42511228)
What is this? Is this a google summarization service?
Re:Exploring LoRA – Part 1: The Idea Behind Parameter Effici... by jwildeboer on Wednesday December 25, 2024 @04:09AM (#42507817)
(Not to be confused with LoRa, (short for long range) which is a spread spectrum modulation technique derived from chirp spread spectrum (CSS) technology, powering technologies like LoRaWAN and Meshtastic)
Re:Exploring LoRA – Part 1: The Idea Behind Parameter Effici... by FusspawnUK on Wednesday December 25, 2024 @05:07AM (#42507996)
really wish they had come up with another name. googling gets annoying
Re:Exploring LoRA – Part 1: The Idea Behind Parameter Effici... by the__alchemist on Wednesday December 25, 2024 @08:27AM (#42508679)
Contributors: They both use mixed capitalization. They have partially-overlapping audiences.
Re:Exploring LoRA – Part 1: The Idea Behind Parameter Effici... by SeasonalEnnui on Wednesday December 25, 2024 @08:49AM (#42508759)
This gets me every time. I expect to see something interesting and it turns to be the other one. One is a fantastic thing and the other is mediocre, pick which way round at your discretion!
Re:Exploring LoRA – Part 1: The Idea Behind Parameter Effici... by sva_ on Wednesday December 25, 2024 @11:32AM (#42509551)
Pretty simple to spot LoRa vs LoRA.
Re:Exploring LoRA – Part 1: The Idea Behind Parameter Effici... by rkagerer on Thursday December 26, 2024 @12:49AM (#42513387)
Memory mnemonic: Capital A for "AI"
Re:Exploring LoRA – Part 1: The Idea Behind Parameter Effici... by pavlov on Wednesday December 25, 2024 @12:23PM (#42509843)
What exactly is the confusion? Does “parameter efficient fine-tuning” mean anything in context of the other Lora? If not, then it’s probably obvious which one this is about.
Re:Exploring LoRA – Part 1: The Idea Behind Parameter Effici... by mrgaro on Wednesday December 25, 2024 @01:19PM (#42510206)
Actually it does: Lora the radio protocol has parameters to tune. Usually both sender and receiver needs to match these, so I read this like a method how these could be automatically tuned based on the distance and radio environment.
Re:Exploring LoRA – Part 1: The Idea Behind Parameter Effici... by danielhanchen on Wednesday December 25, 2024 @04:38AM (#42507904)
Super cool series of articles! :)