News for hackers. Stuff that matters.

A comment on Show HN: Explore large language models on any computer with 512MB of RAM.

Re:Show HN: Explore large language models on any computer wi... by 1024core on Saturday June 17, 2023 @12:09PM (#36371484)
>>> lm.do("If I have 7 apples then eat 5, how many apples do I have?")

'You have 8 apples.'

>>> lm.set_max_ram('4gb')

4.0

>>> lm.do("If I have 7 apples then eat 5, how many apples do I have?")

'I have 2 apples left.'

It's funny how it switched from "You" to "I" when the memory was increased.

[ Parent | on HN ]
Re:Show HN: Explore large language models on any computer wi... by george_808 on Saturday June 17, 2023 @12:50PM (#36371935)
From what I can tell, this doesn’t re-seed the random number generator, so we shouldn’t expect deterministic results. A better test would be to examine the model’s logits, or probabilities of the next token, across different ram sizes.
[ Parent | on HN ]