News for hackers. Stuff that matters.
Re:Re-Evaluating GPT-4's Bar Exam Performance by Rinzler89 on Sunday June 02, 2024 @03:55PM (#40556798)
What does GPT consider being "average wealth and income". Statistics? Or biased weights from anecdotes he formed on the anecdotes he scraped off the internet on how wealthy people say the feel?

Would be cool to know how LLMs shape their opinions.

Re:Re-Evaluating GPT-4's Bar Exam Performance by LeoPanthera on Sunday June 02, 2024 @03:57PM (#40556811)
You can just ask it, you know.

GPT-4o:

“Average wealth and income” can vary significantly by region and context. However, in the United States, as a rough benchmark, the median household income is around $70,000 per year. Wealth, which includes assets such as savings, property, and investments minus debts, is harder to pinpoint but median net worth for U.S. households is approximately $100,000. These figures provide a general idea of what might be considered “average” in terms of wealth and income."

Re:Re-Evaluating GPT-4's Bar Exam Performance by Rinzler89 on Sunday June 02, 2024 @04:05PM (#40556859)
>You can just ask it, you know.

But my question will not be part of the context of that conversation.

Re:Re-Evaluating GPT-4's Bar Exam Performance by LeoPanthera on Sunday June 02, 2024 @04:08PM (#40556876)
Mine was. I asked it the first question, first.
Re:Re-Evaluating GPT-4's Bar Exam Performance by Rinzler89 on Sunday June 02, 2024 @07:48PM (#40558058)
The original conversation I mean, of the grandparent. Me starting an identical conversation will not guarantee an identical context.
Re:Re-Evaluating GPT-4's Bar Exam Performance by lambdaxyzw on Sunday June 02, 2024 @04:26PM (#40556987)
I like that it immediately assumed the US, even though nothing in your question suggested it. I love that all LLMs have a strong US centric bias.

Btw I'm not personally a lawyer, but I've heard that GPT is especially prone to mixing laws across the borders - for example you ask a law question in language X, and get a response that uses a law from a country Y - and it's extremally convincing doing that (unless you're a lawyer, I guess).

Re:Re-Evaluating GPT-4's Bar Exam Performance by fjdjshsh on Sunday June 02, 2024 @05:06PM (#40557190)
This is my experience with Hackernews. If the comment doesn't specify the country, it's an American talking about the USA
Re:Re-Evaluating GPT-4's Bar Exam Performance by LeoPanthera on Sunday June 02, 2024 @04:37PM (#40557057)
ChatGPT has user-customizable "instructions", and mine are set to tell it where I live. Any user can do the same, so that it will not make incorrect assumptions for you.
Re:Re-Evaluating GPT-4's Bar Exam Performance by d3m0t3p on Sunday June 02, 2024 @05:11PM (#40557232)
You might increase the probability of getting a correct answer for your region, but imo you decrease your awareness to allucination. Overall you can still get a wrong answer
Re:Re-Evaluating GPT-4's Bar Exam Performance by LordDragonfang on Sunday June 02, 2024 @04:35PM (#40557043)
I mean, to be fair, if you're speaking English to it, the most likely possibility is that you're inside the US:

https://en.wikipedia.org/wiki/List_of_countries_by_English-s...

I know there's a lot of complaints about things being US-centric, but the US is a very large country.

Re:Re-Evaluating GPT-4's Bar Exam Performance by ordersofmag on Sunday June 02, 2024 @06:28PM (#40557622)
Well, except the number of English speakers outside the US is much larger than inside the US (as per the wikipedia page you point to) by 5 to 1. Granted many folks are speaking it as their 2nd (or nth) language. But when you take into account the limited set of languages supported by ChatGPT one could reasonably assume English-speaking (typing) users of ChatGPT are from outside the U.S. as non-U.S. folks are in the majority of 'folks for whom english would be their first option when interacting with ChatGPT'. Even if you only count India, Nigeria and Pakistan.

Though of course OpenAI can tell (frequently, roughly) where folks are coming from geographically and could (does?) take that into account.

Re:Re-Evaluating GPT-4's Bar Exam Performance by duskwuff on Sunday June 02, 2024 @11:31PM (#40559089)
> but the US is a very large country.

Indeed - the US is a very large country, and consists of over 50 different jurisdictions, each with their own slightly different laws. An answer to a legal question which is correct in one state will often be subtly incorrect in another, and completely wrong in yet another.