News for hackers. Stuff that matters.
Re:Show HN: ChessCoach – A neural chess engine that comments... by cbutner on Tuesday October 05, 2021 @05:34PM (#28765917)
Yeah, that's a massive problem with the natural language domain all across machine learning.

Unfortunately it's very difficult to track down training data for chess commentary in the first place, let alone trim down biases. For reference, I was able to gather about 1 million samples, but it really needs a billion.

Hopefully through data augmentation and better general intelligence models we can make better progress on bias issues soon, as that's a huge problem when we start trusting AI models too much in life.

Re:Show HN: ChessCoach – A neural chess engine that comments... by a_t48 on Tuesday October 05, 2021 @07:12PM (#28766924)
You might be able to kludge a fix to tokenize the output and replace he/him/she/her with them/their. It's not as sexy as the engine outputting the correct words, but it should get the job done.
Re:Show HN: ChessCoach – A neural chess engine that comments... by cbutner on Tuesday October 05, 2021 @08:48PM (#28767742)
Yes, in this case as long as they still agree when it actually names people, I don't think it would be too difficult. There may be factors I'm not considering though.

Harder would be more general models like GPT-2 and GPT-3.

Re:Show HN: ChessCoach – A neural chess engine that comments... by a_t48 on Tuesday October 05, 2021 @08:53PM (#28767765)
Singular "they" doesn't care about the gender of the person named, so it should be good.
Re:Show HN: ChessCoach – A neural chess engine that comments... by prezjordan on Tuesday October 05, 2021 @05:36PM (#28765931)
Appreciate the honesty here. Pretty wild how natural this model feels with 1 million samples.
Re:Show HN: ChessCoach – A neural chess engine that comments... by cbutner on Tuesday October 05, 2021 @05:40PM (#28765986)
Sometimes it seems really accurate (like the cherry-picked GIF in the overview docs) and sometimes really off.

I think for the most part, it knows more than it lets on, but finding the right sampling methods (or better yet, generalized search) to generate the best comments is a tough problem because it's difficult to evaluate quality.

There's some info on the sampling methods here: https://chrisbutner.github.io/ChessCoach/high-level-explanat...