We explore the AI2050 “hard problems” that block the promise of AI and cause AI risks

Evaluating modern ML models is hard. The strong incentive for researchers and companies to report a state-of-the-art result on some metric often leads to questionable research practices (QRPs):

Are large language models (LLMs) biased towards text generated by LLMs over text authored by humans, leading to possible anti-human bias?

Forecasting technologies is difficult; for instance, recent AI progress surprised many of us.

In their 1990 book Uncertainty, Morgan and Henrion investigated experts’ ability to predict the future, concluding that “elicited expert judgments may be seriously flawed, but are often the only game in town.”

Thanks to Misha Yagudin, Eli Lifland, Jonathan Mann, Juan Cambeiro, Gregory Lewis, @belikewater, and Daniel Filan for forecasts.

After recent events in Ukraine, Samotsvety convened to update our probabilities of nuclear war. In March 2022, at the beginning of the Ukraine war, we were at ~0.01% that London would be hit with a nuclear weapon in the next month.

What are the most important risks and precursors of global catastrophic risks each week? We parse millions of news pieces a week to find out.