Will AI be able to read a novel and reliably answer questions about it before 2030?

Metaculus
★★★☆☆
99%
Virtually certain
Yes

Question description

On May 31st, 2022, prominent deep learning skeptic and NYU professor emeritus Gary Marcus challenged Elon Musk to a bet on AGI by the end of 2029. His proposed bet consists of 5 AI achievements, of which he predicted no more than 2 would come to pass before 2030. This question is about Marcus' second prediction,

In 2029, AI will not be able to read a novel and reliably answer questions about plot, character, conflicts, motivations, etc. Key will be going beyond the literal text, as Davis and I explain in Rebooting AI.

For this challenge, we will use the NarrativeQA dataset as an illustrative example of a benchmark that could trigger positive resolution,

To encourage progress on deeper comprehension of language, we present a new dataset and set of tasks in which the reader must answer questions about stories by reading entire books or movie scripts. These tasks are designed so that successfully answering their questions requires understanding the underlying narrative rather than relying on shallow pattern matching or salience.

Indicators

IndicatorValue
Stars
★★★☆☆
PlatformMetaculus
Number of forecasts759

Capture

Resizable preview:
99%
Virtually certain

On May 31st, 2022, prominent deep learning skeptic and NYU professor emeritus Gary Marcus challenged Elon Musk to a bet on AGI by the end of 2029. His proposed bet consists of 5 AI achievements, of which he predicted no more than 2 would come to pass...

Last updated: 2024-04-24
★★★☆☆
Metaculus
Forecasts: 759

Embed

<iframe src="https://https://metaforecast.org/questions/embed/metaculus-11177" height="600" width="600" frameborder="0" />