Five years after AGI, will a ‘long reflection’ be underway?

Metaculus
★★★☆☆
8%
Very unlikely
Yes

Question description

Some effective altruists, including Toby Ord and William MacAskill, have argued that, if humanity succeeds in eliminating existential risk or reducing it to acceptable levels, it should not immediately embark on an ambitious and potentially irreversible project of arranging the universe's resources in accordance to its values, but ought instead to spend considerable time— "centuries (or more)"; "perhaps tens of thousands of years"; "thousands or millions of years"; "[p]erhaps... a million years"—figuring out what is in fact of value. The long reflection may thus be seen as an intermediate stage in a rational long-term human developmental trajectory, following an initial stage of existential security when existential risk is drastically reduced and followed by a final stage when humanity's potential is fully realized.<br/> —EA Forum Wiki

Note that more recent thinking suggests a long reflection aided by advanced AI could happen much faster than the timeframes cited above:

Original discussion of the long reflection indicated that it could be a lengthy process of 10,000 years or more. More recent discussion [...] i) takes seriously the possibility that the long reflection could last just weeks rather than years or millenia, and ii) notes that wall clock time is probably not the most useful way to think about the length of the reflection, given that the reflection process, if it happens at all, will likely involve many superfast AIs doing the bulk of the cognitive labor.<br/> —Aldred, 2024, fn. 4

(Related question: “Five years after AGI, will AI philosophical competence be solved?”)

Indicators

IndicatorValue
Stars
★★★☆☆
PlatformMetaculus
Number of forecasts36

Capture

Resizable preview:
8%
Very unlikely

Some effective altruists, including Toby Ord and William MacAskill, have argued that, if humanity succeeds in eliminating existential risk or reducing it to acceptable levels, it should not immediately embark on an ambitious and potentially...

Last updated: 2024-10-07
★★★☆☆
Metaculus
Forecasts: 36

Embed

<iframe src="https://https://metaforecast.org/questions/embed/metaculus-26363" height="600" width="600" frameborder="0" />