News for hackers. Stuff that matters.
Re:Fine Tuning Mistral 7B on Magic the Gathering Draft by dharmab on Tuesday December 05, 2023 @03:32PM (#38536531)
That lines up with my experience- for high-stakes decisions, they rarely give me a great answer. But for low stakes decisions, they do well at giving me a good enough answer. For example, I've been using them to help find gifts for friends and children this month. I don't need the best choice to solve the problem, just a good one.
Re:Fine Tuning Mistral 7B on Magic the Gathering Draft by pixl97 on Tuesday December 05, 2023 @03:58PM (#38536813)
How much additional calculation occurs in high-stakes decisions by individuals. Also what is the variability in quality of high stakes decisions in humans?

I'm guessing LLM decision is rather average, but that the LLM has no easy way of spending the extra time to gather information around said high stakes decisions like a human would.

Re:Fine Tuning Mistral 7B on Magic the Gathering Draft by s1artibartfast on Tuesday December 05, 2023 @07:19PM (#38538843)
I dont think additional calculation is the difference. It makes more sense to think of individual humans as models which are highly tuned.

Just like like LLMs, some humans are better tuned than others for specific tasks, as well as in general.

Re:Fine Tuning Mistral 7B on Magic the Gathering Draft by nicbou on Wednesday December 06, 2023 @04:53AM (#38542109)
The difference is that you can reject a low-stakes answer that's invalid. You can tell that something is off, or it doesn't matter.

With high-stakes decisions, you're surrendering the decision-making power to the AI because you don't understand the output well enough to verify it.

Basically, and AI can give ideas but not advise.

Re:Fine Tuning Mistral 7B on Magic the Gathering Draft by nothrowaways on Tuesday December 05, 2023 @06:08PM (#38538204)
What are examples of low stakes
Re:Fine Tuning Mistral 7B on Magic the Gathering Draft by dharmab on Tuesday December 05, 2023 @08:18PM (#38539336)
Generating content for tabletop gaming with my friends (especially wacky ideas, like character names themed after items on the Taco Bell menu)

I had to buy some spare tools where I cared more about price than quality and it helped me choose some suitable brands

As mentioned, you can tell it a bit about a person (and feed in their wishlist if they have one) and it'll help you pick something they'll probably like

Finding something to do to spend an afternoon in a city while traveling

In general, anything where there is no objective best answer (meaning I can ask it to generate multiple possibilities and filter out the bad ideas) and where I value speed over correctness.

Re:Fine Tuning Mistral 7B on Magic the Gathering Draft by entropicdrifter on Wednesday December 06, 2023 @01:27PM (#38547703)
> Generating content for tabletop gaming with my friends (especially wacky ideas, like character names themed after items on the Taco Bell menu)

I've been going ham with this. Pay for ChatGPT Plus for work, so GPT-4's been helping me design encounters, plan sessions, and brainstorm ideas for one-shots. It gives me a sort of bland, vanilla idea for something, I suggest a twist on it, it goes, "oh great idea, here that is with your changes:" and I iterate with it from there.

Likewise I love theming characters, plotlines, and settings after songs, bands and albums, so I'll dump in a bunch of lyrics and ask ChatGPT to help me intertwine subtle references into descriptions, names, and plot points.

Re:Fine Tuning Mistral 7B on Magic the Gathering Draft by rictic on Tuesday December 05, 2023 @06:55PM (#38538652)
A random sampling of things GPT-4 has helped me with lately:

Where are the dates in whole foods? (A: with nuts, not fruits and veggies)

How can I steam bao without a steamer basket? (A: saucepan, 1" water, balled up aluminum foil, plate, baos, lid)

Any guess as to when this photo was taken? It looks like anywhere from the 70s to the 90s. (A: the photo paper has a logo that postdates a 2003 company merger)

Re:Fine Tuning Mistral 7B on Magic the Gathering Draft by maypop on Wednesday December 06, 2023 @07:47AM (#38543135)
They can’t do super human performance like alpha go and the can’t think “system 2” that would be required for high stakes decisions.*

Room to grow.

*observations from the recent karpathy llm talk.