News for hackers. Stuff that matters.
Re:Abstract Wikipedia by crazygringo on Thursday July 02, 2020 @02:24PM (#23716361)
I think a consistent multilingual Wikipedia is a fantastic goal.

But I'm not sure this is the right way to do it.

Given that most of the information on Wikipedia is "narrative", and doesn't consist of facts contained in Wikidata (e.g. a history article recounting a battle, or a movie article explaining the plot), this scope for this will be extremely limited. The creators are attempting to address this by actually containing every single aspect of a movie's plot as a fact, and that sentences are functions that express those facts... but this seems entirely unwieldy and just too much work.

What I've wished for instead, for years, is actually an underlying "metalanguage" that expresses the vocabulary and grammatical concepts in all languages. Very loosely, think of an "intermediate" linguistic representation layer in Google Translate.

Obviously nobody can write in that directly in a user-friendly way. But what you could do is take English (or any language) text, do an automated translation into that intermediate representation, then ask the author or volunteers to identify all ambiguous language cases" -- e.g. it would ask if "he signed" means made his signature, or communicated in sign language. It would also ask for things that would need clarification perhaps not in your own language but in other languages -- e.g. what noun does "it" refer to, so another language will know to use the masculine or feminine version. All of this can be done within your own language to produce an accurate language-agnostic "text".

Then, out of this intermediate canonical interpretation, every article on Wikipedia would be generated back out of it, in all languages, and perfectly accurately, because the output program isn't even ML, it's just a straight-up rule engine.

Interestingly, an English-language original might be output just a little bit different but in ways that don't change the meaning. Almost like a language "linter".

Anyways -- I think it would actually be doable. The key part is a "Google Translate"-type tool that does 99% of the work. It would need manual curation of the intermediate layer with a professional linguist from each language, as well as manually curated output rules (although those could be generated by ML as a first pass).

But something like that could fundamentally change communication. Imagine if any article you wanted to make available perfectly translated to anyone, you could do, just with the extra work of resolving all the ambiguities a translating program finds.

Re:Abstract Wikipedia by SkyBelow on Thursday July 02, 2020 @03:41PM (#23717161)
>The creators are attempting to address this by actually containing every single aspect of a movie's plot as a fact, and that sentences are functions that express those facts... but this seems entirely unwieldy and just too much work.

Doesn't this also get into issues where facts aren't clearly defined. I can think of a lot of interpretation of meaning from my literature classes, but there are also questions such as ownership of land at contested borders, if something was a legal acquisition or theft, or even coming up with a factual distinction between when something is grave robbery vs archaeology. A personal favorite would be mental illness, especially with some of the DSM V changes that have largely been rejected (or outright ignored) by society. And there are all sorts of political disagreements.

And as this applies to different languages, and different languages are likely aimed at different cultures and different nations, this gets messy. I could see some differences in an article written in Hindi vs Chinese when concerning issues involving both China and India. Creating a common language will force a unification of these differences that currently might exist with a sort of stalemate with each linguistic side by maintained by the dominant country for that language.

Re:Abstract Wikipedia by yorwba on Thursday July 02, 2020 @05:51PM (#23718495)
> questions such as ownership of land at contested borders

The entity representing Kashmir https://www.wikidata.org/wiki/Q43100 has three statements each for "country" and "territory claimed by", reflecting the claims by China, India and Pakistan. There are separate entities for Taiwan (the Republic of China) https://www.wikidata.org/wiki/Q865 , Taiwan (the island) https://www.wikidata.org/wiki/Q22502 , Taiwan (the province of the Republic of China) https://www.wikidata.org/wiki/Q32081 and Taiwan (the province of the People's Republic of China) https://www.wikidata.org/wiki/Q57251 .

So Wikidata can handle conflicting information by collecting all of it, but clearly separating the different viewpoints in a kind of "split brain". That works so long as the different sides can agree that their opponent's views are what they state they are.

In an Abstract Wikipedia article, that means that all viewpoints with a sufficiently large userbase might end up represented equally in all language versions, but they'll still be clearly distinguished as such, so the reader can apply their own value judgments to support their own side's viewpoint over that of their enemies.

Re:Abstract Wikipedia by majewsky on Thursday July 02, 2020 @07:35PM (#23719299)
So Abstract Wikipedia could achieve the ultimate goal of being banned in China, India and Pakistan at the same time.
Re:Abstract Wikipedia by 20after4 on Friday July 03, 2020 @02:57AM (#23721679)
And maybe even in the USA.
Re:Abstract Wikipedia by IfOnlyYouKnew on Thursday July 02, 2020 @04:34PM (#23717713)
Ambiguity, probability, and conflict can themselves be modeled.

As long as participants agree on something resembling a common reality, they should at least be able to agree that there is a disagreement.

Re:Abstract Wikipedia by schoen on Thursday July 02, 2020 @07:12PM (#23719137)
This reminds me greatly of Leonard Cohen's line

"There is a war between the ones who say there is a war and the ones who say there isn't."

https://www.youtube.com/watch?v=1qjmxTQ1o0M

(In Wikipedia jargon, I guess some people or countries are apt to say that their opponents' points of view are non-notable?)

Re:Abstract Wikipedia by amirouche on Thursday July 02, 2020 @02:32PM (#23716442)
> But what you could do is take English...

Then it is not english but a subset of english. They are translation systems that work the way you describe already using restricted natural language grammars.

Re:Abstract Wikipedia by crazygringo on Thursday July 02, 2020 @02:39PM (#23716516)
To the contrary, it would be a superset. The point is that it is not restricted, but would also require additional distinctions that aren't even present in English.

Although, there may be some cases where truly identical meanings in English are collapsed into one. For example, the arguable lack of difference between "it isn't" and "it's not". (Things that are quotations could me marked up as such to not disturb the text in its original language, but quotations could still be translated into other languages.)

Re:Abstract Wikipedia by sitkack on Thursday July 02, 2020 @04:59PM (#23718011)
Even if you wrote the articles in a subset of English like the https://simple.wikipedia.org/wiki/Main_Page and then used ML to translate into other languages and then formed a feedback loop with the translation so that the original author could have some assurance that the translated texts were valid, this would be huge.
Re:Abstract Wikipedia by noema on Thursday July 02, 2020 @06:55PM (#23719028)
>What I've wished for instead, for years, is actually an underlying "metalanguage" that expresses the vocabulary and grammatical concepts in all languages.

We've been down this road before... https://en.wikipedia.org/wiki/Characteristica_universalis

Re:Abstract Wikipedia by crazygringo on Thursday July 02, 2020 @07:53PM (#23719427)
Thanks for the reference. What I'm talking about is quite different, though.

Leibniz's proposal and similar ones are about a language intended to represent knowledge (formal concepts), and usable by people (with training).

I'm talking about simply a translation layer. It doesn't "know" or symbolize anything on its own -- it's merely a central mapping between all constructs in different languages. Nobody would read or write in it directly, and it couldn't be used to make inferences or anything like that.

What Abstract Wikipedia is proposing is actually quite similar to what Leibniz proposed, an intricately detailed map of knowledge and true propositions.

What I'm proposing is vastly less ambitious and vastly more useful: just a way to store the meaning of text in an intermediate representation that belongs to no single language. But that representation doesn't bother with what are facts or propositions or anything like that -- just entities and relations that mapped to however we ultimately understand languages in the real world. And assisted with a ton of ML, rather than built up from elementary propositions or set theory or something.

Re:Abstract Wikipedia by noema on Saturday July 04, 2020 @01:35PM (#23733696)
I think ultimately this endeavor would run into the same issues which befell the efforts of the GOFAI research program. A translation layer would need to "understand" embedded higher-order contexts of language making implicit references to the world, and which vary by different languages and cultures. Such contexts are not analytically decomposable in a manner which permits a centralized mapping.

As with GOFAI, a formalized representation of meaning entails certain ontological assumptions about "how the world hangs together". These assumptions often hold true for an isolated domain of inquiry but falter when dealing with more complex phenomena. I could, however, see such a system useful for 'Simple Wikipedia' articles which largely consist of existentially quantifiable sentences.

There is a wealth of philosophical scholarship on the issues of semantics and formal representation; I recommend the works of Hubert Dreyfus if you want to dig into this more.

Re:Abstract Wikipedia by jes5199 on Friday July 03, 2020 @02:45AM (#23721629)
hasn’t it been suggested that the internals of google translate and similar systems have effectively constructed such a meta-language?

It seems like you could construct a meta-linguistic sentence by writing a phrase in one language and choosing from possible translations in additional languages, which would hone the cross-linguistic meaning. You might not even have to define a representation of the metalanguage, you could just keep a set of (phrase, score) tuples, and interpolate translations of those phrases into the target language at display time! that almost sounds practical to me

Re:Abstract Wikipedia by noema on Saturday July 04, 2020 @01:37PM (#23733711)
>hasn’t it been suggested that the internals of google translate and similar systems have effectively constructed such a meta-language?

No, Google Translate and other state of the art machine translation software rely on large corpuses of linguistic data from which they make statistical inference. They do not have a 'centralized metalanguage' which ontologizes meaning in the manner described by OP.

Re:Abstract Wikipedia by miracle2k on Friday July 03, 2020 @05:36AM (#23722525)
Is the Grammatical Framework something like the thing you are imagining?