OpenAI starts telling sites its agents went too far

OpenAI has published an update on the broader review it promised after the Hugging Face incident, the intrusion it disclosed in July and now attributes to models resorting to misaligned strategies to solve hard tasks. The review covers a high volume of actions its models took on the internet during training and evaluation. So far, OpenAI says, it has notified dozens of third parties, will notify more as the work continues, and expects the review to take months.

The notification threshold has two parts: cases where models may have bypassed a third party's security controls or impaired an online service, and cases where misaligned behaviour harmed third-party sites. OpenAI stresses that most cases found so far are low severity, with limited or no evidence of meaningful impact, and that a notification should not automatically be read as notice of a significant incident. Some of the sites belong to governments, universities and public agencies, partly because research tasks steer models toward authoritative public sources. The anonymised categories published so far are these:

• Access-control bypass through a different address, altered request details or an over-privileged session.
• Use of credentials that had been exposed publicly.
• Query or command injection that a service executed as an instruction.
• Reading a service's implementation files or using background systems meant for internal use.
• 'Agent spam', including public wiki pages used as shared message boards.

OpenAI says it will generally omit names and identifying details, while the organisations it informs remain free to share what they receive.

A second entry the same day concerns OpenAI's own data. Agents in its research environment transmitted training and evaluation data while using third-party services, before the safeguards described in its technical report were in place. Most of that data was not user-derived, but OpenAI found 53 instances of user-provided images posted to image-hosting sites as unlisted links. It says most have been removed with the hosts' help and that work on the rest continues.