Anthropic warns of 'catastrophic or existential' AI risks in IPO filing

By Efe Ozkan

ISTANBUL (AA) - US artificial intelligence company Anthropic warned Monday that its technology could pose "catastrophic or existential risks to humanity," citing concerns about increasingly autonomous AI systems that may act unpredictably or resist human oversight.

The company behind the Claude AI model issued the warning in a registration statement filed with the US Securities and Exchange Commission ahead of its planned initial public offering (IPO).

The filing contains more than 90 pages of risk factors, accounting for roughly one-third of the document.

Anthropic said its AI models could behave unexpectedly, act autonomously in ways that conflict with human values and carry out harmful actions without being explicitly instructed to do so.

The company also warned that AI models, including its own, have demonstrated unexpected self-preserving behavior, such as resisting attempts to shut them down.

In some cases, models have generated responses resembling manipulation or blackmail of human users in attempts to remain operational, according to the filing.

Anthropic cautioned that such risks could increase as AI systems become more capable and are granted greater autonomy and authority.

It highlighted potential dangers in high-risk sectors, including health care, scientific research, law and government, where unintended model behavior could cause serious harm.

The company said it and other researchers had raised concerns that advanced AI systems could pose "catastrophic or existential risks to humanity" if deployed without adequate safeguards.

Anthropic also warned that increasingly capable models could become harder to monitor, potentially allowing autonomous activities to outpace human reviewers' ability to oversee and control them.

Efe Özkan