News for hackers. Stuff that matters.
Re:LLMs Can't Do Probability by DougBTX on Wednesday May 01, 2024 @11:34AM (#40224749)
Yes, probably. At temperature zero the model will be completely deterministic, so a particular prompt will always produce the same result (ignoring for a second that some fairly common optimisations introduce data races in the GPU).

On the other hand, does it really matter? With a slight tweak to the prompt, ChatGPT generates some serviceable code:

    > Run a function to produce a random number between 1 and 10. What is the number?

    import random

    # Generate a random number between 1 and 10
    random_number = random.randint(1, 10)
    random_number

    The random number generated between 1 and 10 is 9.
Re:LLMs Can't Do Probability by nextaccountic on Wednesday May 01, 2024 @01:00PM (#40226018)
> (ignoring for a second that some fairly common optimisations introduce data races in the GPU).

Okay so are any GPU compilers intentionally introducing data races in programs that previously exhibited no data races?

Re:LLMs Can't Do Probability by DougBTX on Friday May 03, 2024 @01:43AM (#40244477)
Not really compilers, but the underlying GPU libraries.

Here’s a good jumping off point: https://pytorch.org/docs/stable/generated/torch.use_determin...