MetaforecastStatus
SearchToolsAbout

‌

‌
‌
‌
‌
‌
‌

Will loss curves on Pythia models of different sizes trained on the same data in the same order be similar?

Manifold Markets
★★☆☆☆
77%
Likely
Yes

Question description #

Someone in the EleutherAI discord is reporting that finetuning Pythia models of different sizes on the same data in the same order is giving spookily similar loss curves, just vertically shifted.

[image]Will training Pythia models from scratch in the same way produce similar behaviour? Resolves N/A if it turns out the original result was just a bug or something like that.

Indicators #

IndicatorValue
Stars
★★☆☆☆
PlatformManifold Markets
Forecasters13
VolumeM803

Capture #

Resizable preview:
Will loss curves on Pythia models of different sizes trained on the same data in the same order be similar?
77%
Likely
Last updated: 2025-04-09

Someone in the EleutherAI discord is reporting that finetuning Pythia models of different sizes on the same data in the same order is giving spookily similar loss curves, just vertically shifted.

[image]Will training Pythia models from scratch in...

Last updated: 2025-04-09
★★☆☆☆
Manifold Markets
Forecasters: 13
Volume: M803

Embed #

<iframe src="https://metaforecast.org/questions/embed/manifold-rfydmwU98t6vesHbiW4P" height="600" width="600" frameborder="0" />

Preview