OpenAI warns that its new ChatGPT Agent has the ability to aid dangerous bioweapon development

• OpenAI has warned that its new ChatGPT Agent feature poses heightened bioweapon risks. The tool is designed to automate everyday tasks, but safety researchers found the model could also potentially help novices create biological threats. The launch comes amid growing competition to build AI agents capable of autonomously completing tasks.

OpenAI’s newest product promises to make it easier for someone to automatically gather data, create spreadsheets, book travel, spin up slide decks—and, just maybe, build a biological weapon. ChatGPT Agent, a new agentic AI tool that can take action on a user’s behalf, is the first product OpenAI has classified as having a “high” capability for biorisk.

This means the model can provide meaningful assistance to “novice” actors and enable them to create known biological or chemical threats. The real-world implications of this could mean that biological or chemical terror events by non-state actors become more likely and frequent, according to OpenAI’s “Preparedness Framework,” which the company uses to track and prepare for new risks of severe harm from its frontier models.

“Some might think that biorisk is not real, and models only provide information that could be found via search. That may have been true in 2024 but is definitely not true today. Based our evaluations and those of our experts, the risk is very real,” Boaz Barak, a member of the technical staff at OpenAI, said in a social media post.

“While we can’t say for sure that this model can enable a novice to create severe biological harm, I believe it would have been deeply irresponsible to release this model without comprehensive mitigations such as the one we have put in place,” he added.

OpenAI said that classing the model as high risk for bio-misuse was a “precautionary approach,” and one that had triggered extra safeguards for the tool.

Keren Gu, a safety researcher at OpenAI, said that while the company did not have definitive evidence that the model could meaningfully guide a novice to create something of severe biological harm, it had activated safeguards nonetheless. These safeguards include having ChatGPT Agent refuse prompts that could potentially be intended to help someone produce a bioweapon, systems that flag potentially unsafe requests for expert review, strict rules that block risky content, quicker responses to problems, and robust monitoring for any signs of misuse.

One of the key challenges in mitigating the potential for biorisk is that the same capabilities could unlock life-saving medical breakthroughs, one of the big promises for advanced AI models.

The company has become increasingly concerned about the potential for model misuse in biological weapon development. In a blog post last month, OpenAI announced it was ramping up safety testing to reduce the risk of its models being used to aid in the creation of biological weapons. The AI lab warned that without these precautions, the models could soon enable “novice uplift”—helping individuals with little scientific background develop dangerous weapons.

“Unlike Nuclear and Radiological threats, obtaining materials is less of a barrier for creating bio threats and hence security depends to greater extent on scarcity of knowledge and lab skills,” Barak said. “Based on our evaluations and external experts, an unmitigated ChatGPT Agent could narrow that knowledge gap and offer advice closer to a subject matter expert.”

ChatGPT Agent

OpenAI’s new ChatGPT feature is an attempt to cash in on one of the buzziest, and most risky, areas of AI development: agents.

The new feature functions like a personal assistant, capable of handling tasks such as booking restaurant reservations, online shopping, and organizing job candidate lists. Unlike previous versions, the tool can use a virtual computer to actively control web browsers, interact with files, and navigate across apps like spreadsheets and slide decks.

The company merged the teams behind Operator, its first AI agent, and Deep Research, a tool developed to conduct multi-step online research for complex tasks, to form a single group that developed the new tool.

AI labs are currently racing to build agents that can manage complex digital tasks independently, and the launch follows similar releases by Google and Anthropic. Big Tech companies see AI agents as a commercial opportunity, as companies are increasingly moving to implement AI into workflows and automate certain tasks.

OpenAI has acknowledged that greater autonomy introduces more risk and is emphasizing user control to mitigate these risks. For example, the agent asks for permission before taking significant action and can be paused, redirected, or stopped by the user at any time.

This story was originally featured on Fortune.com