Classic example:

    The iron ball fell on the glass table, and it shattered.
    The glass ball fell on the iron table, and it shattered.

Human readers will pick the correct antecedent for "it" in each case, so it's not ambiguous. But correct interpretation depends on knowing something about how likely glass and iron are to shatter.

I love these kinds of examples, but I don't think they ought to significantly deter the kind of work shown here; it depends on the application, but basic interpretation could be very useful even if it can't handle every case.

These are called Winograd Schemas; you can read more about them and see more examples at https://www.cs.nyu.edu/davise/papers/WS.html

Yeah, it's a bit of a nitpick over definition of "comprehension" but the program itself is obviously useful, I mean it's a poor man's version of Freebase. Freebase/Google Knowledge Graph doesn't claim to "comprehend" anything, it is just a large graph datastructure with an efficient querying mechanism. That's what this is (...given this is a poor man's version, efficient querying mechanism may be lacking for larger graphs)

We don't have to worry about the philosophical questions of comprehension to find cases like this present challenges. These types of example (better articulated in mrec's link) are cases where we can imagine straightforward, plausible queries that the computer would be unable to answer: "what shattered?" or "what remains?" Some of the examples are such that simple web searches wouldn't be helpful in resolving the ambiguity; you'd need some "common knowledge." (Again: not that that's an insurmountable or necessarily relevant problem.)

For those curious, this process is called "anaphora resolution", which is the process of understanding what a reference(usually a pronoun) actually refers to.

While this is a interesting point, grammarians will bristle at such an imprecise use of 'it', which should unambiguously refer to one antecedent.

Who are these 'grammarians', and why should anyone care what makes them bristle? They evidently have a complete lack of appreciation for the semantic content of sentences and can only see syntax. What a very poor view of language that would be.

Professional writers, editors, and teachers of language would be perfectly fine with those sentences, because they understand language as being made up of syntax AND semantics.

In the admittedly artificially constructed SAT, the second sentence would seem to be an almost classic example of unacceptable ambiguity. Perhaps in real life, you are correct that the adjectives provide sufficient context.

OpenCyc is a really cool implementation of one solution to this problem.

The way it was explained to me, computers don't know that when you turn a full cup of coffee upside down, the liquid will fall out. Cyc attempts to provide a framework for that.

http://sw.opencyc.org

> Bobby picked up the toy. Then he put down the toy.

When we read this sentence, our brains automatically augment additional information based on the verb. However, in this example, my program will fail to answer because my program does not augment any additional information but it can be extended to.

This can be implemented in our program by created a new property for each object called "location". If a verb is location based, we can set the location of the object based on what the verb describes. For example, "the toy"'s location could be "Bobby's hands" after the first sentence based on the verb phrase "pick up". So the program will understand where the toy is and be able to understand queries related to "where".

As you can imagine, implementing this would be very tedious since there are too many cases for all the verbs. My program may not be able to do advanced reading comprehension (reading between lines and augmenting information) but I argue that it can do simple reading comprehension, in that it can understand the relationship between objects. There is still a long way to go before my program is capable of more sophisticated reading comprehension, but in theory, I think my approach seems possible.

there is a linguistic research called "usage based" and also "emergent grammar" that i find interesting regarding subjects like this:

1. meaning is usage. 2. structure emerges from usage.

meaning we don't use grammar to produce language, it is an emergent property (irreducible) of "pre-linguistic pragmatics" ...