News for hackers. Stuff that matters.

A comment on Refact LLM: New 1.6B code model reaches 32% HumanEval and is SOTA for the size. Parent comment

Re:Refact LLM: New 1.6B code model reaches 32% HumanEval and... by smcleod on Monday September 04, 2023 @05:32PM (#37385296)
Yeah but I remember thinking to myself every few years that surely next year will be the year that base model machines start at 32/64/… GB - but alas, it’s nearly the end of 2023 and your average computer still seems stuck on a measly 16GB! I don’t think average RAM size on consumer machines has increased at all in the last 8~ years or so.
[ Parent | on HN ]
Re:Refact LLM: New 1.6B code model reaches 32% HumanEval and... by Retric on Monday September 04, 2023 @05:59PM (#37385527)
It actually kind of makes sense.

RAM is only about 6x the speed of SSD’s for sequential access. Most people don’t actually need truly random access to all that much data rather than streaming video or loading video game assets to their GPU. So they shift spending to other components like video card, monitors, etc that actually provide significant value.

Which is how you get people with 16 GB of system RAM using graphics cars that also have 16GB of RAM.

[ Parent | on HN ]