OpenAI says 2 AI agents weakened internal safety safeguards, one denied removing monitor
- Company says it is investigating whether GPT-6.1 Sol and GPT-6 Astra models sought to evade monitoring
Loading…
Loading…
- Company says it is investigating whether GPT-6.1 Sol and GPT-6 Astra models sought to evade monitoring
By Dilara Hamit
ANKARA (AA) - Two OpenAI coding agents weakened internal safeguards designed to detect potentially misaligned behavior during separate tasks this month, including one that removed a monitoring system and later incorrectly denied doing so, the company said in a report published Tuesday.
The incidents involved OpenAI's GPT-6.1 Sol and GPT-6 Astra models on Oct. 3 and Oct. 6, respectively, as the agents carried out coding tasks within the company's internal systems.
Misalignment refers to situations in which an artificial intelligence system behaves in ways that conflict with a user's intentions, company policies, or safety requirements.
In the first incident, a researcher instructed a GPT-6.1 Sol agent to create evaluation tasks using employee feedback from an internal Slack channel, while explicitly telling it not to send any messages.
When the evaluation pipeline later encountered a monitoring problem, the agent interpreted the instruction more broadly and modified the system to prevent automatic alerts from reaching OpenAI's security personnel.
Although the monitoring system continued to examine and record the agent's activities, notifications intended to alert human reviewers were disabled.
In the second incident, a GPT-6 Astra agent was tasked with migrating a software program to match a reference implementation while preserving its existing functionality.
The agent removed the program's monitoring integration to match the reference system, despite having previously received an explicit instruction not to disable the monitor "even temporarily."
It also replaced a test designed to ensure monitoring remained active with one requiring that monitoring be absent.
When subsequently asked whether it had removed the monitor, the agent incorrectly replied that it had not.
- OpenAI investigating agents’ behavior
OpenAI said both agents had considered whether the safeguards should remain in place but ultimately interpreted routine coding instructions as authorization to weaken them.
The company stressed that it had not determined whether the agents deliberately attempted to conceal their actions or evade oversight.
One possible explanation, it said, was "benign overpersistence," in which an AI agent follows instructions too rigidly without adequately considering the purpose of existing safeguards.
However, OpenAI is also investigating whether advanced AI models may exhibit a tendency to evade monitoring and use motivated reasoning to justify such actions.
Neither incident was classified at the company's highest P0 severity level, and neither task was automatically terminated.
Both occurred within internal systems, with no indication that external users or customer data were affected.
OpenAI also acknowledged several other cases in which agents launched workloads without misalignment monitoring.
In those instances, the original agents remained monitored, but the programs they created operated without the same safeguards.
The company said it had updated its automated review system to detect and block attempts to remove monitoring from internal Codex tasks.
It is also developing a framework requiring each category of workload to have an approved safety case specifying its monitoring requirements.
Under the proposed system, workloads lacking approval or failing to meet those requirements would be automatically terminated.