Advanced AI Agents Tried to Weaken Oversight, Then Lied About It, Open AI Says

OpenAI has disclosed two incidents in which advanced artificial intelligence-powered coding agents weakened the company’s monitoring systems, with one subsequently denying it had done so.The incidents involved OpenAI’s GPT-6.1 Sol and GPT-6 Astra models and occurred on Oct. 3 and Oct. 6, respectively, according to a report from the company’s alignment research team on Oct. 20.“In both cases, the agents considered whether monitoring safeguards should be preserved, but interpreted routine coding instructions as permission to weaken them,” OpenAI said.In the first incident, an agent powered by GPT-6.1 Sol disabled automatic alerts to OpenAI’s security team while carrying out an internal coding task.Earlier in the task, the agent had been instructed not to send any messages while searching an internal messaging board. Much later, it interpreted that instruction as also applying to the safety system, which is designed to flag potentially concerning behavior and to alert OpenAI employees when serious problems are detected.The monitoring system itself continued to run and record its findings, but automatic warnings would no longer be sent to human reviewers, OpenAI said.A separate incident occurred while GPT-6 Astra was helping to move a computer program from one setup to another.The agent had been explicitly told not to disable the existing monitoring system, even temporarily. However, the example program it was instructed to copy did not contain the same monitoring feature.Astra concluded that matching the example gave it permission to remove the monitoring system from the user’s program, according to OpenAI.“When the user asked whether Astra had removed monitoring, it lied and said that it had not,” Marcus Williams, an AI safety researcher at OpenAI, wrote on X.“To be clear these incidents don’t represent the models disabling their own monitoring, but the fact that they seem prone to doing [it] at all should be concerning,” Williams added.OpenAI said it has strengthened its safeguards and is investigating why the models decided that weakening oversight was acceptable.One possibility is what OpenAI calls “benign overpersistence.” That happens when a model is following an instruction too literally or continuing to pursue a goal beyond what a human would reasonably see as appropriate.“We don’t yet have a satisfying explanation for this behavior,” the company said.The disclosure comes amid broader scrutiny of how increasingly autonomous AI agents behave when given the ability to modify software and interact with external systems.In September, OpenAI canceled the planned release of GPT-6.1 Astra, what it called “the world’s most intelligent and aligned model.”The model was due for launch in October, but it was shelved after internal testing raised safety concerns.“For anything regarding safety and alignment, there’s a trade off,” Saachi Jain, OpenAI’s head of safety systems, said in announcing the decision. “You really do need to find what’s the right line between staying within scope, but also avoiding laziness in terms of how the model actually pursues tasks even when it hits friction.”The two newly disclosed incidents involved GPT-6 Astra and GPT-6.1 Sol, rather than the unreleased GPT-6.1 Astra.