Using mmap to make LLaMA load faster

Re:Using mmap to make LLaMA load faster by IAmNotACellist on Wednesday April 05, 2023 @01:59PM (#35458068)

That's not the original PR. jart was working on a malloc() approach that didn't work and slaren wrote all the code actually doing mmap, which jart then rebased in a random new PR, changed to support an unnecessary version change, magic numbers, a conversion tool, and WIN32 support when that was already working in the draft PR. https://archive.ph/Uva8c

This is the original PR: https://github.com/ggerganov/llama.cpp/pull/586.

Jart's archived comments:

"my changes"

"Here's how folks in the community have been reacting to my work."

"I just wrote a change that's going to let your LLaMA models load instantly..."

https://archive.ph/PyPFZ

"I'm the author"

https://archive.ph/qFrcY

"Author here..."

"Tragedy of the commons...We're talking to a group of people who live inside scientific papers and jupyer notebooks."

"My change helps inference go faster."

"The point of my change..."

"I stated my change offered a 2x improvement in memory usage."

https://archive.ph/k34V2

"I can only take credit for a 2x recrease in RAM usage."

https://archive.ph/MBPN0

"I just wrote a change that's going to let your LLaMA models load instantly, thanks to custom malloc() and the power of mmap()"

https://archive.ph/yrMwh

slaren replied to jart on HN asking her why she was doing and saying those things, and she didn't bother to reply to him, despite replying to others in that subthread within minutes. https://archive.ph/zCfiJ

[ Parent | on HN ]

Re:Using mmap to make LLaMA load faster by empathy_m on Wednesday April 05, 2023 @02:56PM (#35458843)

Hmm, based on what you've quoted here and knowing nothing else but a few messages on AI Twitter I would invest in jart.

This is BillG-style product skill -- there is a ton of work that goes into representing a piece of software as something important and valuable that people should buy into.

[ Parent | on HN ]

Re:Using mmap to make LLaMA load faster by smoldesu on Wednesday April 05, 2023 @04:42PM (#35460107)

Jart is a pretty exceptional engineer, even if she wrote this patch single-handedly it would hardly be a footnote in her list of professional accomplishments. This is the author of Cosmopolitan libc, redbean and APE we're talking about, after all.

That being said, it's important to attribute work properly. It can be easy to mix things up (eg. "my patch" is excusable) but repeatedly insisting authorship when you're not the author of the change just seems disingenuous. I'm sure it was in good faith, but since they didn't address the issue or clear anything up, it's come to this.

Dramatic, and hardly the conclusion people wanted to the story of a free performance improvement. It's not entirely contrived though, and I think the maintainer handled this exceptionally well given the circumstances.

[ Parent | on HN ]

Re:Using mmap to make LLaMA load faster by zamnos on Thursday April 06, 2023 @10:56AM (#35469174)

> This is the author of Cosmopolitan libc, redbean and APE we're talking about, after all.

Is this? If she so easily misrepresented slarens work as hers in this case, what other work isn't actually attributable to jart?

[ Parent | on HN ]

Re:Using mmap to make LLaMA load faster by smoldesu on Thursday April 06, 2023 @01:08PM (#35471323)

I'm all for detracting from suspicious authors, but it's unlikely Justine just steals their code wholecloth. She's been an active community member for a while, and wrote a lot of impressive software before LLMs and script kiddies democratized the whole process.

In this specific instance, jart had a communication error that she failed to clarify, and so things compounded from there. The part that she didn't author is clearly defined in Git, and the most-plausible explanation is an honest mistake. Assuming ill-intent requires you to ignore the original context of the disagreement and focus on the outrage, which pretty much says it all.

That being said, I'd love to hear what evidence you have to the contrary. Maybe you've got a link to an FTP server from 2001 with the Blinkenlights source code on it, I can't say for sure. A fraud probably doesn't write in-depth patch breakdowns on their personal blog for fun, though.

[ Parent | on HN ]