Background

Humanity's Last Exam (HLE) is a benchmark designed to test AI models at the frontiers of human expertise. The exam consists of expert-level questions across various fields, deliberately crafted to be extremely challenging. Current AI models have performed poorly on this benchmark, with leading models answering fewer than 10% of expert questions correctly.

Resolution Criteria

This market will resolve YES if any AI model achieves a verified score of 80% or higher on Humanity's Last Exam before April 1, 2025. The score must be:

Independently verified by Scale AI or another reputable organization

Achieved on the full exam, not a subset

Publicly announced and documented

Achieved through a single model's capabilities (not through combining multiple models or human assistance)

The market will resolve NO if no AI model achieves a verified score of 80% or higher by April 1, 2025.

Considerations

The current performance gap between AI models (<10%) and the target (80%) is substantial

Experts predict models might exceed 50% accuracy by the end of 2025, making an 80% score by April 2025 particularly ambitious

The exam is specifically designed to test the limits of AI capabilities, making rapid improvements more challenging than on typical benchmarks

Scale AI's methodology and scoring criteria may evolve, but resolution will be based on their official scoring system at the time of evaluation