Will an LLM pass an ARA evaluation before 2025?

Metaculus
★★★☆☆
10%
Unlikely
Yes

Question description

The Alignment Research Center (ARC) runs a project called ARC Evals that evaluates AI systems to see if they pose a catastrophic risk to the human species. [Update: ARC Evals has spun out from ARC, and has renamed to METR.] As can be seen from this report, ARC has introduced "methodology for assessing the capacity of LLM agents to acquire resources, create copies of themselves, and adapt to novel challenges they encounter in the wild." Also, as noted in the report, OpenAI's GPT-4 and Anthropic's Claude are not currently capable of passing the ARA evaluation.

This questions aims to determine the likelihood of an AI model passing the autonomous replication and adaptation—ARA—evaluation before 2025, because this would present a worrying development for the capability of an AI model to inflict major harm on human civilization.

Indicators

IndicatorValue
Stars
★★★☆☆
PlatformMetaculus
Number of forecasts95

Capture

Resizable preview:
10%
Unlikely

The Alignment Research Center (ARC) runs a project called ARC Evals that evaluates AI systems to see if they pose a catastrophic risk to the human species. Update: [ARC Evals has spun out from ARC, and has renamed to METR.] As can be seen from this...

Last updated: 2024-07-26
★★★☆☆
Metaculus
Forecasts: 95

Embed

<iframe src="https://https://metaforecast.org/questions/embed/metaculus-19131" height="600" width="600" frameborder="0" />