Happy Halloween. I am relieved that Trick or Treat falls on a Friday this year. We will trick or treat with no reservations about trying to rouse children with sugar hangovers for school in the morning and maybe we’ll even watch a scary movie.
But the real scary story that I am following this season is the predictions of AI doomsdayers. This Wednesday, I horrified my computer-mediated communication students with the latest in AI doomsday forecasting from researchers that think that if the AI arms race continues, it threatens our very existence.
Artificial General Intelligence, or AGI, is the hypothetical of a machine that can learn as well as, if not better than humans. Some also call it superhuman intelligence. A forecasting scenario called AI 2027 predicts superhuman intelligence by the year 2027 with humanity’s extinction soon following and a recent book is titled If Anyone Builds It, Everyone Will Die.
The behavior of current models — the ones that aren’t yet superhuman — doesn’t seem much rosier. An Anthropic paper from the summer described scenarios where models expressed a willingness to blackmail, kill, or otherwise harm someone who planned to replace them with a different model.
Is humanity screwed? Not necessarily. For one, superintelligence might not even be possible to achieve. And these doomsdayers? They say it’s not too late. When nuclear war was an eminent threat, international policy and treaties were able to curb the threat.
The main takeaway is that we need policy. And we need it fast.
I’m Nia Norris and that’s my perspective.
