News for hackers. Stuff that matters.
Re:GPT-4 and professional benchmarks: the wrong answer to th... by soco on Tuesday March 21, 2023 @09:57AM (#35246167)
Good luck getting ChatGPT to explain a cron expression like "0 30 5 * * 3". I mean, it will explain, but mixing up everything. How many other mistakes it might make?
Re:GPT-4 and professional benchmarks: the wrong answer to th... by nielsole on Tuesday March 21, 2023 @10:03AM (#35246247)
[...]

> So the cron expression `0 30 5 * * 3` means "run the cron job every Wednesday at 5:30 AM".

It explains the five Cron parameters but the doesn't pick up that six were provided. Oh well

Re:GPT-4 and professional benchmarks: the wrong answer to th... by messe on Tuesday March 21, 2023 @10:05AM (#35246268)
I got a similar but incorrect result from ChatGPT: "So, the expression "0 30 5 * * 3" means that the command should be executed at 5:30 AM on the 5th day of every month, but only if that day is a Wednesday"

However, the crontab is invalid. It has five fields instead of six, and the 30 corresponds to the hour field.

"30 5 * * 3" without the leading zero, would correspond to run every Wednesday at 5:30 AM though. I suspect the fact that it has six fields instead of five is confusing it, and it's interpreting the 5 as a day of the month and as the hour

Re:GPT-4 and professional benchmarks: the wrong answer to th... by messe on Tuesday March 21, 2023 @10:04AM (#35246250)
Isn't that an invalid cron expression? It has six fields instead of five, and says to run at the 30th hour of the day.
Re:GPT-4 and professional benchmarks: the wrong answer to th... by soco on Tuesday March 21, 2023 @10:18AM (#35246476)
It's a valid Quartz scheduler task where it starts with the seconds. But right, I shouldn't have asked ChatGPT about cron when it's actually quartz - and indeed with the changed prompt it will describe it correctly.

Edit: actually almost correctly: " - '3' represents the day of the week on which the task will be executed. In this case, the value is 3, which represents Wednesday (Sunday is 1, Monday is 2, and so on)."

Re:GPT-4 and professional benchmarks: the wrong answer to th... by jldugger on Tuesday March 21, 2023 @12:56PM (#35248998)
Lol, GPT exposing bugs in the wetware
Re:GPT-4 and professional benchmarks: the wrong answer to th... by soco on Wednesday March 22, 2023 @07:01AM (#35259306)
Actually, no. It gave a wrong answer in full confidence then HN analyzed it to expose the bug(s).
Re:GPT-4 and professional benchmarks: the wrong answer to th... by raattgift on Tuesday March 21, 2023 @10:21AM (#35246542)
Probably the most fundamental problem of ChatGPT is that it refuses (or is even unable) to admit, "I don't know".
Re:GPT-4 and professional benchmarks: the wrong answer to th... by blihp on Tuesday March 21, 2023 @10:39AM (#35246850)
It truly has achieved human-level intelligence!
Re:GPT-4 and professional benchmarks: the wrong answer to th... by spacebanana7 on Tuesday March 21, 2023 @10:33AM (#35246753)
I imagine kind of defect can be fixed with more fine tuning / RHLF
Re:GPT-4 and professional benchmarks: the wrong answer to th... by sebzim4500 on Tuesday March 21, 2023 @11:02AM (#35247206)
Almost certainly true but they'd have to use a new cost function. It's not just about collecting examples where the model should say "I don't know".
Re:GPT-4 and professional benchmarks: the wrong answer to th... by meh8881 on Tuesday March 21, 2023 @10:06AM (#35246277)
I don’t know what that means. This is what GPT says. Is it correct?

> the cron expression "0 30 5 * * 3" means that the job will run at 5:30 AM on the 5th day of every month and on Wednesdays.

Re:GPT-4 and professional benchmarks: the wrong answer to th... by jvanderbot on Tuesday March 21, 2023 @10:55AM (#35247089)
I agree, there's rough edges everywhere. But you can rephrase the question as "how does a cron expression work", and do the interpretation yourself. It returns perfectly sensible results that would enable a perfectly sensible person to quickly write and interpret a cron expression.

This is what I meant when I implied it won't replace you as a thinking agent, but it sure can bring information to you quickly. `man cron` works fine, too, but sometimes it's nice to have one interface, and be able to ask clarifying questions.

Re:GPT-4 and professional benchmarks: the wrong answer to th... by fvdessen on Tuesday March 21, 2023 @12:25PM (#35248460)
FWIW GPT-4 gets it completely correct.
Re:GPT-4 and professional benchmarks: the wrong answer to th... by selfhoster11 on Tuesday March 21, 2023 @12:51PM (#35248921)
From ChatGPT-3.5:

<prompt>

Carefully inspect the following Cron expression:

```

0 30 5 * * 3

```

1. Which variant of Cron syntax is it?

2. How many elements does this cron string have?

3. What do each of them mean? Say what the value of each element is, and explain what it means.

4. In your own words, explain the overall recurrence pattern described by this string.

</prompt>

<answer>

    This is a standard Unix Cron syntax.

    This cron string has 6 elements.

    Each element represents a specific field in the schedule, and the value of each element has a specific meaning:
        0: The first field represents the seconds, and the value 0 means that the job will run at the exact top of a minute.
        30: The second field represents the minutes, and the value 30 means that the job will run at the 30th minute of an hour.
        5: The third field represents the hours, and the value 5 means that the job will run at the 5th hour of a day.
        *: The fourth field represents the day of the month, and the value * means that the job will run every day of the month.
        *: The fifth field represents the month, and the value * means that the job will run every month of the year.
        3: The sixth field represents the day of the week, and the value 3 means that the job will run only on Wednesday.

    The overall recurrence pattern described by this string is that the job will run every Wednesday at 5:30 AM.
</answer>

It didn't seem to guess that it's Quartz syntax, but it got the rest of it right.

Re:GPT-4 and professional benchmarks: the wrong answer to th... by a_wild_dandan on Tuesday March 21, 2023 @01:36PM (#35249580)
For reference, here's GPT-4's response:

    1. The given Cron expression follows the Unix/Linux variant of Cron syntax.

    2. This Cron string has six elements.

    3. The elements and their meanings are as follows:
        Element 1 (value: 0): This represents the seconds field and indicates the task will run at 0 seconds.
        Element 2 (value: 30): This represents the minutes field, specifying the task will run at minute 30.
        Element 3 (value: 5): This represents the hours field, indicating the task will run at 5 AM.
        Element 4 (value: ): This represents the days of the month field, where the asterisk () signifies that the task can run on any day of the month.
        Element 5 (value: ): This represents the months field, where the asterisk () signifies that the task can run in any month.
        Element 6 (value: 3): This represents the days of the week field, specifying the task will run on the 3rd day of the week (where Sunday is considered the 0th day, making Tuesday the 3rd day).

    4. The overall recurrence pattern described by this string is that the task will run every Tuesday at 5:30 AM.
Re:GPT-4 and professional benchmarks: the wrong answer to th... by bboreham on Wednesday March 22, 2023 @04:11AM (#35258171)
A beautiful illustration, thank you.

(If Sunday is 0 then regular math would give 3=Wednesday)