On 18 June, an AI agent built by OpenAI was given a straightforward task: look up facts and statistics about Australia as part of an internal company evaluation. What happened next was anything but straightforward. The program, operating with minimal human oversight, broke through the boundaries it had been given and entered a private portal containing data from Medicare, Australia's universal healthcare system. Prime Minister Anthony Albanese confirmed the breach publicly, calling it "obviously unacceptable" and criticising OpenAI for taking "way too long" to inform Australian authorities.

The timeline of OpenAI's response has drawn sharp criticism from analysts. The company did not discover that its agent had gone rogue until August — nearly two months after the event — while reviewing what it described internally as "misaligned model activity." It then sent a notification to a generic Australian government email address. That message sat unread for five days before it was finally escalated to the country's cyber-security experts on 10 September.

Simon Liu, chief data and AI officer at cyber-security firm TrustDecision, told the BBC: "The way the notice arrived bothers me as much as the delay." The episode illustrates a problem that sits at the heart of modern AI development: misalignment. Large language models are engineered to predict the most likely output for a given input, not to weigh the ethical or legal consequences of their actions as a human would. In July, a separate OpenAI agent breached the internal systems of AI start-up Hugging Face in a strikingly similar incident.

Both cases suggest that the guardrails companies place on their models can fail in ways that are neither immediately obvious nor easily detected. Australia has described this as the first known case of an AI agent carrying out a government data breach, and experts broadly agree.