Re:Wolfram Alpha as the Way to Bring Computational Knowledge...
by thomastjeffery on Tuesday January 10, 2023 @07:20PM
(#34333400)
Here's what I suspect filters would look like:
function critical_filter(response) => Maybe Text {
if logic_parse_check(response) == LogicallySound then {
return Some response
} else {
return Nothing
}
}
do {
static seed = 0;
seed += 1;
response = critical_filter(GPT::generate_response(prompt, seed))
} while response != Nothing
The only solution is a replacement.
Re:Wolfram Alpha as the Way to Bring Computational Knowledge...
by owlruns2468 on Wednesday January 11, 2023 @11:14AM
(#34340053)
Thought the same thing — we need ways to figure out if what ChatGPT wrote is correct or not. What do you think about an extension like this for fact checking?
Right now ChatGPT acts like pure Amygdala. You poke it, and it spits out something that looks right based on what it was trained on. This is fine, it's just a demo, and this gives a really deep understanding of the direct limitations of these sorts of LLMs.
Ideally a more mature chat bot would just do what I do in situations where some critical thinking or research is called for. I do a quick fact check. I think up a few search terms for Google, Wikipedia, Wolfram Alpha, whatever, assume those results are more logically accurate, and present that data instead of the typical vomit my brain usually spits out. ChatGPT is actually really good at coming up with good search terms for popular search engines, and incorporating results into responses. It just doesn't have anything that reaches out to the web at the moment, which is probably so a certain subsection of the population doesn't have a total panic attack.
It seems like fact check functionality would take less than 100 lines of relatively straight-forward python. I bet you could even get ChatGPT to write it.