How far ahead we look when parsing and understanding text

Question

Part of trying to understand how we mentally parse and understand text requires understanding how far ahead we look, which is what this question is about.

I'm wondering how we understand how to interpret words that have multiple totally unrelated meanings and/or pronunciations. Words such as:

read (red, reed)
tear (tair, teer)

If you see a sentence like:

And the tears were many.

It's ambiguous. You have to backtrack a few times and try out different meanings of tears. If it was part of more text, then you could maybe figure it out.

They fell hard and ruined their clothes. They were running so fast. It was in the middle of the night. The tears were many.

Still ambiguous.

They fell hard and ruined their clothes. The tears were many.

Still ambiguous.

They fell hard and ruined their clothes. The tears on their clothes were many.
// or
The clothes were shredded. The tears were many.

Then we can finally see that it is "tairs". The crazy thing is how we can read that out loud and get it right. The question is, how can we pronounce that right in the first case, where the meaning is clarified after the word is used, and in the second case, where the meaning is clarified before.

It is as if we see a box of text, a window of text.

Then we can finally see that it is "tairs". 
     _____________________________
The |crazy thing is how we can rea|d that out 
    |                             |
loud| and get it right. The questi|on is, how 
    |                             |
can |we pronounce that right in th|e first case, 
    |                             |
wher|e the meaning is clarified _a|fter_ the 
    |                             |
word|is used, and in the second ca|se, where 
     ‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾
the meaning is clarified _before_.

The current word there would be that, and it's like we can interpret that far in advance, or more, to determine how to say the current word. But this doesn't seem to act in parallel, but instead just happens in series really fast.

The question is, if there is any research or formal approaches that explain how this process works in humans.

Answer 1

These are more or less like the word-sense disambiguation, anaphora resolution or co-reference resolution examples in the Winograd Schema Challenge and generally in natural language understanding.

How far ahead do we look when parsing and understanding text?

As you essentially show in your examples where the necessary information is not in the sentence, the information needed for understanding can be arbitrarily far away, and not necessarily ahead.

For example in the article headline The Pope's Baby Steps on Gays [1], where and when did you find pieces of information you use to decide that Steps is N not V?

That's why these tasks are arguably AI-hard and inarguably require a lot of local and global context as input, although we can get much of the way with cruder methods.

any research or formal approaches that explain how this process works in humans

This is unsolved. The examples require logic and reasoning. If we knew how it worked and could describe it formally, we could implement full AI.

In the top literature on this cluster of problems from the computer science perspective, I have not found any direct reference to linguistic or biological research. We tend to take human intelligence as a given. But it could be that if you follow the references they themselves make reference to such research in other fields. There is research on biological neural networks, usually focussed on low-level functions.

1 - example stolen from Chris Manning's 2016 talk

Answer 2

Since you seem to be interested in reading, you may find the following overview of "The science of word recognition" (written in 2017, by Kevin Larson) to be an interesting read.

Larson says that the best supported model of how words are read is parallel recognition of multiple letters.

Larson also talks about the phenomenon of eye "saccades", which provides experimental support for non-linearity in the way people read sentences/paragraphs. I don't know if the distinction you mention between "parallel" and "in series really fast" is actually important.

For information about theories about how people read aloud, one relevant article is "Modeling Reading: The Dual-Route Approach", by Max Coltheart. Coltheart says that words with "irregular" spellings are read aloud more slowly than words with regular spellings. Frequency also has an effect.

Answer 3

A well-known and popular formal model for human language is CFG (meaning "context free phrase structure grammar") described by Chomsky in the '50s. CFG is my own personal favorite grammatical theory for human language. Here is a useful document describing how look ahead works in a standard C-language parser, Bison (developed from the classic Yacc) works.