Alarmist Intelligence

When gods say exhilaration may end in extinction, the world takes a pause. Dario Amodei of the AI behemoth Anthropic periodically strays from the pantheon and pronounces his commandments as a creator with a conscience larger than his creation. Early this year, in a long essay titled ‘The Adolescence of Technology’, he cautioned, in the dual roles of optimist and realist, humanity about the risks AI could pose not only from the labs but in the hands of rogue governments as well. “The years in front of us will be impossibly hard, asking more of us than we think we can give. But in my time as a researcher, leader, and citizen, I have seen enough courage and nobility to believe that we can win—that when put in the darkest circumstances, humanity has a way of gathering, seemingly at the last minute, the strength and wisdom needed to prevail. We have no time to lose,” he wrote.

Now he thinks we are losing time. In another essay, ‘We Must Pace the Frontier’, the philosopher-king of AI tells us that the risks of an AImperium—he doesn’t use the word—are as serious as its benefits. “We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain,” he warns. What worries him most is AI’s capability to build its next generations without human involvement, or what is now called recursive self-improvement. Translation: The monster doesn’t require a Frankenstein as father any more. He is becoming so intelligent that a post-humanity planet of machines is not mere fantasy, so slow down before you may even be denied a chance to slow down. Add to his argument for restraint the outright alarmism—an AI-inflicted end to the human race is possible within a decade, to take the most cited example—and what we get in the place of deliverance is doomerism.

We have never been here before: Humans do all the talking; machines deny them a vocabulary of comprehension and coherence—with the exception of perhaps one human named Donald Trump who believes that there is nothing to worry about as long as you have a president smarter than the Altmans and Amodeis. The talking, increasingly, is about how to take back control—or how to preserve the science about the future as a human enterprise. The talking is powered by a morality born from existential anxieties, as if the infinity of invention has made the inventor feel suddenly small. As if Autonomous Intelligence won’t tolerate human ambition beyond a point. And the doings of the machine have already made tomorrow an excitement with a subtext of terror.

This sort of Endism filled the conversation when it was ideology that wrote heaven and built terror, and its provenance, too, was from the human quest for alternatives. History may not have come to a standstill with the fall of communism and the triumph of liberal democracy, and the struggle for alternatives continued, the only difference being that the Book of men was replaced by the Book of God. The ecstasy of afterlife, the very essence of martyrdom, was equalled by the horror of here and now. Technology is certainly not miming, in its evolution and ambition, ideology and faith. In the arguments it has spawned, the idea of heaven defies consensus, gods lose their halo.

The doomsayers are the creators themselves, which is a huge shift in the history of alternatives. The cult of the code maker is built around a few wise men; and the choices they have given to the rest constitute freedom in its most exhilarating sense. It is that freedom now seen as a threat to humanity. The raging conversation, with the tech lords all united in “pacing the frontier”, portrays AI as an entity with an inscrutable mind. They have not downgraded their divinity as the last masters of the universe. They alone will be the arbiters—and set the limits of machine-imagination. The conversation clouds the obvious: It is not the powerlessness of humanity in the face of omniscient intelligence they are highlighting. It is their own powerlessness in maintaining a balance between the commercial race to the top and the nature of the science they are selling.

The terror of the alternative, human-induced though it is, does not necessarily nullify the original intent—or the abundance of creativity that has made it possible in the beginning. The arc of AI, no matter how much awe and apprehension it continues to generate, mirrors the limitlessness of not technology but the human mind. Only promptings by the reckless, whether from the free world or dictatorships with lots of chips in their armoury, can make the artificial real. We are not there yet.