Re:Re-Evaluating GPT-4's Bar Exam Performance
by fjdjshsh on Sunday June 02, 2024 @05:06PM
(#40557190)
This is my experience with Hackernews. If the comment doesn't specify the country, it's an American talking about the USA
Re:Re-Evaluating GPT-4's Bar Exam Performance
by LeoPanthera on Sunday June 02, 2024 @04:37PM
(#40557057)
ChatGPT has user-customizable "instructions", and mine are set to tell it where I live. Any user can do the same, so that it will not make incorrect assumptions for you.
Re:Re-Evaluating GPT-4's Bar Exam Performance
by d3m0t3p on Sunday June 02, 2024 @05:11PM
(#40557232)
You might increase the probability of getting a correct answer for your region, but imo you decrease your awareness to allucination.
Overall you can still get a wrong answer
Re:Re-Evaluating GPT-4's Bar Exam Performance
by LordDragonfang on Sunday June 02, 2024 @04:35PM
(#40557043)
I mean, to be fair, if you're speaking English to it, the most likely possibility is that you're inside the US:
https://en.wikipedia.org/wiki/List_of_countries_by_English-s...
I know there's a lot of complaints about things being US-centric, but the US is a very large country.
Re:Re-Evaluating GPT-4's Bar Exam Performance
by ordersofmag on Sunday June 02, 2024 @06:28PM
(#40557622)
Well, except the number of English speakers outside the US is much larger than inside the US (as per the wikipedia page you point to) by 5 to 1. Granted many folks are speaking it as their 2nd (or nth) language. But when you take into account the limited set of languages supported by ChatGPT one could reasonably assume English-speaking (typing) users of ChatGPT are from outside the U.S. as non-U.S. folks are in the majority of 'folks for whom english would be their first option when interacting with ChatGPT'. Even if you only count India, Nigeria and Pakistan.
Though of course OpenAI can tell (frequently, roughly) where folks are coming from geographically and could (does?) take that into account.
Re:Re-Evaluating GPT-4's Bar Exam Performance
by duskwuff on Sunday June 02, 2024 @11:31PM
(#40559089)
> but the US is a very large country.
Indeed - the US is a very large country, and consists of over 50 different jurisdictions, each with their own slightly different laws. An answer to a legal question which is correct in one state will often be subtly incorrect in another, and completely wrong in yet another.
Btw I'm not personally a lawyer, but I've heard that GPT is especially prone to mixing laws across the borders - for example you ask a law question in language X, and get a response that uses a law from a country Y - and it's extremally convincing doing that (unless you're a lawyer, I guess).