Re:Tell HN: Copying and pasting from ChatGPT unsolicited sucks
by Der_Einzige on Wednesday December 07, 2022 @01:44PM
(#33898353)
Btw, I wrote a whole paper about language models and their ability to follow linguistic, semantic, or phonetic constraints.
I bet you'll like it.
https://paperswithcode.com/paper/most-language-models-can-be...
This is why it's easy to make ChatGPT reveal bias, and why the restrictions patching that bias tend to fall prey to simple ruses that reveal its preferences; everything looks like a norm to it, so it lacks the causal chains that would lead towards logical outcomes. As a result it's extremely gullible and emits nonsense when given a tough logic question.
Given a task like "give me a list of words ending in the letter u", it will oblige with a very lengthy alphabetical list of words, most of which end in u, but not all. Asked to find the largest set of rhyming words in the list, its answers changed radically each time, from "I can't do that" to a somewhat plausible candidate(except for having words that don't end in u) to getting stuck repeating "buttocks".
I used it to write some business language to respond to a recruiter. It's decent at being a secretary, since that stuff is 99% norms.