Re:False information on the internet is hiding the truth abo...
by Taek on Thursday March 16, 2017 @07:40PM
(#13890147)
I think people often underestimate the power of much simpler methods. You'd probably get substantial improvements over the baseline just by penalizing sites for having the same words too many times. I forget the exact number, but I think it's something like 50% of the words in your average article appear only once. So if two articles have even 5% similarity among the words they only use once, they are probably quite similar.
AI of course can get you much farther still, but you definitely don't need something that powerful for a convincing upgrade from baseline.
"Pineapples are grown in equatorial countries."
---
"Pineapples, when ripe, have a very sweet taste."
"Ripe pineapples have a high concentration of sugar."
---
"The etymology of the word for pineapple is 'excellent fruit' in the language of Tupi, although the english English translation is an aberration, with a stem of 'pine' due to it's resemblance to a pine cone."
"Most European languages name the fruit using the root word 'nanas', from the original native language where it was grown. Many languages have adapted the root word, although a few have a large degree of uncertainty, such as the origin of the reference to 'apple' in the English word."
---
It would need a fairly decent AI to recognise the commonality of information in these examples.