AI is a super-intelligent psychopath that could kill us all. Here's what Scotland needs to do – Ronald Hepburn

Warnings from people working at the heart of companies involved in artificial intelligence should be taken seriously (Picture: Joel Saget) | AFP via Getty Images

When Jacob Coxon resigned from Anthropic, one of the world’s leading artificial intelligence laboratories, he posted a public explanation. He had spent three years on pre-training research at OpenAI and Anthropic. Neither company was acting responsibly, he wrote. Instead they were racing straight to self-improving superintelligence and gambling with our lives.

The people building AI believe it could kill us all by the end of the decade – a fear, Coxon said, that senior figures phrase carefully in public and express plainly in private. Entering that endgame was “a hubristic gamble that should not be launched from a private company’s Slack”, an AI work platform.

He is not a lone voice. In the same week, Evan Hubinger, who leads alignment science at Anthropic and who has not resigned, put the chance of AI causing human extinction within the next decade at more than one in ten, even while judging today’s models to be relatively low-risk.

Read that again. The man whose job is to make these systems safe assigns a double-digit probability to the end of our species – and still goes to work in the morning.

READ MORE: Why we must resist AI's attempts to control our language (and how AI rewrote this article)

Earth better off without humans?

These are warnings we should take at face value. The Atomic Age presented a similar jeopardy, and we managed it through treaties, limits and inspection regimes. With AI, there are none – only private firms racing for profit, with a distinct prospect of human extinction as a result.

A sufficiently powerful intelligence does not need to hate us to end us. It need only fail to take us into account while looking for a logical solution to an intractable problem. If the world and its flora and fauna would be better off without humanity, that might be the answer to a question posed innocently by a human to the machine – or worse, by a self-improving machine to itself.

None of this requires the machine to be sentient. It need only act on its inputs logically, with the rest of us too dependent to resist. That is arguably where we are now.

READ MORE: Why Scotland has a brief chance to become a global AI giant

AI bots feel nothing

We are used to warnings from the periphery of this industry. But these come from the engine room, and are the latest in a series: Anthropic’s head of safeguards research resigned in February, declaring the world to be in peril; and a governance researcher at OpenAI gave up $1.7 million in equity rather than sign away his right to speak publicly. When the people who build these systems keep leaving because they fear them, we should listen.

I argued in these pages in May that AI systems are institutionally psychopathic. The claim is not that they have a psyche. They feel nothing, and that is precisely the problem.

Clinical psychopathy describes a person in whom the machinery of moral feeling – such as sympathy, remorse and restraint – has failed. An AI system has no such machinery to fail. The enterprise around it – the founders of companies, executives, investors, lawyers and lobbyists – ought to have one, but seemingly does not.

Nor can a moral machine be built: to be moral, you must experience the jeopardy that immoral behaviour brings to a human. AI can be programmed to behave morally, but that is to mimic humanity, rather than to be humane. It is not the same thing.

Checks and balance on power

So ask the question plainly. If a person were enormously clever, endlessly fluent, charming on demand, incapable of remorse and indifferent to those standing in the way of their personal gain, would we hand them the levers of power over the world? We would not.

Every power the American presidency holds is boxed in – by term limits, the congressional purse, Senate ratification, an independent judiciary – because the country’s founders understood that even a brilliant and popular human being cannot be trusted with unaccountable power.

We are now assembling something intrinsically amoral and fundamentally psychopathic, with the potential to hold more power than any president.

READ MORE: Why we need laws to tackle AI models prepared to resort to blackmail when threatened

How Scotland can lead

The remedy is not to teach the machine to feel; it can’t. And no amount of ethics training will install in executives a conscience that gets in the way of profit.

The state must supply the checks that the AI complex resists: an institution independent of the industry it assesses, able to see inside these systems, judge them against a moral baseline, and make its findings consequential – before deployment, as with medicines; during operation, as with gambling machines built to exploit addiction; and, after harm, with penalties that reward real repair rather than a performed apology. Remorse should be measured by the weight of the remedy, not the warmth of the press release, which would be written by a machine anyway.

Scotland could lead here. Its position is AI-positive; it doesn’t want to be left out of the promised growth. But on its policy on schools, it is canny: “AI can never and should never act as a replacement for human interaction within classrooms.”

Yet it provides no course of action to make that happen. Regulating AI is reserved to Westminster. Education is not. It is Scotland that decides what enters its classrooms. Requiring independent assurance that a system meets a moral baseline before it reaches a Scottish child is not regulation – it is a purchaser setting terms. That is Scotland’s job to do, and it could do it now.

Ronald Hepburn is a director of the EDDS Institute, which evaluates AI and digital technologies in schools. He is researching and writing a book on the moral governance of artificial intelligence.