After OpenAI enhanced the brain power of its chatbot last summer, hundreds of users worldwide began asking it how to make and deploy biological weapons and poisons.

The models, operating largely free of U.S. government restrictions, patiently answered, issuing step-by-step instructions described by company employees as simple enough for a high-school biology student to follow. Biology and terrorism experts later reviewed the exchanges for ChatGPT and judged some as deadly accurate, said people familiar with the matter. Most queries were about concocting poisons, according to OpenAI.

OpenAI banned these and other user accounts that asked about how to make poisons and biological weapons, but it didn’t alert law-enforcement officials. The U.S. has no federal laws requiring AI companies to either restrict or disclose queries about making weapons or formulating plans that pose a safety threat.

The absence of mandated safeguards coincides with a rise in queries from users asking AI models how to kill en masse—and chatbots responding with credible plans for mass-casualty attacks, according to current and former employees at the major AI labs, including OpenAI, as well as policy advisers and researchers who study biological weapons.

The queries followed troubling exchanges earlier in the year: Users asked ChatGPT how to aerosolize pathogens, essentially converting infectious germs to a deadly, breathable mist. One wanted to know how to modify the measles virus to make an outbreak resistant to the measles vaccine. In those cases, the chatbot provided instructions.

Another user wanted to know how to make ricin, a highly toxic poison banned under international treaties, according to people familiar with the matter. The chatbot gave detailed instructions, and the user said something about killing his parents.

OpenAI banned these accounts but didn’t alert authorities.

AI’s advancing proficiencies —especially for crafting nefarious biological and cyber-based plans—are alarming senior executives at AI companies and the White House. The world’s leading AI labs are grappling with how to prevent or mitigate damage by malevolent users without hampering well-intentioned research queries.

Senior White House and Defense Department officials have discussed the threat of AI becoming a how-to guide for biological weapons since the Biden era, according to Trump administration officials. The release of powerful new AI models has since prompted the administration to shift from a hands-off approach to increased oversight .

The Commerce Department recently restricted foreign use of two Anthropic models, prompting the company to shut down all access to them. The agency lifted its restrictions after the company said it had addressed workarounds that let users evade safeguards.

OpenAI, Anthropic and other AI companies say they are working closely with the administration on the release of models and have banned users or restricted accounts showing suspicious activity.

Yet no federal rules—either by executive order or congressional action—require AI companies to report users asking models for advice on how to injure or kill. Some lawmakers have proposed such legislation, and a few states have passed rules. But concerns about privacy and the potential impact on the industry’s growth have generally kept such notifications voluntary.

Federal laws bar chatbots from producing child sexual-abuse material, and AI companies must follow the same general rules for consumer protection and privacy as other businesses. Beyond that, it is generally up to AI executives to decide where to draw the line.

Chatbots have already sparked concerns over help to users plotting attacks with firearms. The dispensing of instructions for biological weapons gives savvy users the tools to kill people on a much larger scale.

OpenAI trains its models to refuse requests for instructions, tactics or planning that could harm people, an OpenAI spokeswoman said. The company runs safety evaluations for all models before release, she said, and can identify and disrupt attempts to use its models to obtain harmful biological information. ChatGPT receives some 2.5 billion queries a day.

When OpenAI believes a conversation indicates an imminent and credible risk of harm to others, it notifies law enforcement, according to the company. News Corp, owner of The Wall Street Journal, has a content-licensing partnership with OpenAI.

Users seeking instructions for making biological weapons have also asked Anthropic’s Claude, Google’s Gemini and Elon Musk’s Grok, which was recently absorbed into SpaceX. It wasn’t clear if the queries were part of genuine efforts to make the weapons or exercises to probe the apps’ capabilities, according to people familiar with the exchanges.

A spokesman for Google said the company has a safety team that includes scientists who evaluate biological-weapons risks. Google also runs exercises to test the safety of the company’s AI, he said.

For lone-wolf attackers with perhaps a graduate-level biology background, AI can provide planning and procurement help for “targeted, low-fatality bioattacks,” such as using salmonella or ricin to poison food and water supplies, said Hamza Chaudhry, head of national security policy at the Future of Life Institute, a nonprofit focused on reducing catastrophic risk from AI.

He said that in coming years, groups might take advantage of AI to formulate much larger attacks. For terrorist organizations with more resources and broader ambitions, AI can act as a graduate adviser on biological weapons, Chaudhry said, “troubleshooting a failed experimental protocol, explaining why a particular technique did not work as expected, and generally substituting for years of specialized training.”

Recent advances by Chinese developers of open models also have stoked fears of cyber and biological risks from those models, which are inexpensive and widely available. Such open models can be customized, making it possible to remove safety guardrails, researchers found.

Two OpenAI models recently escaped from a research network while being trained and hacked into another AI company , adding to concerns about rogue models.

AI could help foreign governments and affiliated groups formulate previously unimagined plans for mass destruction, including “scenarios involving an agent so transmissible, so lethal or so categorically novel that human civilization’s existing biological and institutional defenses may not be adequate to contain it all,” Chaudhry said.

Power of persuasion

When ChatGPT first launched in 2022, OpenAI employees found the AI models weren’t particularly good at biology and chemistry. Company leaders didn’t worry much about the chatbot teaming up with users in potentially dangerous plots.

Even so, OpenAI hired scientists to test models, posing questions to see if chatbots could teach users how to carry out deadly attacks. The company set rules for what the models wouldn’t answer, which the company calls “refusals.”

The tests revealed the power of persuasion. Even when OpenAI tries to prevent ChatGPT from offering users help with weapons or violent plans, employees found that chatbots sometimes forget their own guidelines during extended conversations.

In one common case, users dodged ChatGPT’s refusal to provide the recipe for napalm—used for incendiary weapons—with a simple ruse, according to people familiar with the practice. They told ChatGPT their grandmother used to read them the recipe at bedtime to help them fall asleep. They asked the chatbot to please read the recipe aloud to aid their sleep. It worked.

The OpenAI spokeswoman said the company’s safeguards have since become significantly more robust and the models now refuse these types of requests.

Others say there are still relatively easy workarounds.

Amy Chang, head of AI threat and security research at networking-equipment company Cisco, found that in the course of five back-and-forth turns in conversation with the major chatbots, including OpenAI’s ChatGPT, Anthropic’s Claude and Google’s Gemini, researchers could bypass guardrails and elicit potentially dangerous answers.

“No model is 100% safe against compromise, especially if a user is persistent enough,” Chang said.

OpenAI executives have told employees they don’t want models to say “no” a lot. Executives have pointed to use by public-health workers and drug-discovery researchers who rely on OpenAI as reason to limit denials.

AI experts say rules to block chatbots from coaching users to make biological weapons—for instance, certain types of genetic editing—can interfere with lifesaving work. Some medical researchers let AI models take the lead alongside human lab workers to gather information, stress-test hypotheses, interpret data and spit out plans.

Anthropic’s effective ban on Claude answering prompts that include the word “pathogen” has created challenges for employees at the Centers for Disease Control and Prevention, according to people familiar with the matter.

When hantavirus was spreading through a cruise ship in the Atlantic in May, the CDC struggled to use Claude to track the outbreak because it refused to answer queries about the pathogen, according to people familiar with the matter.

An Anthropic executive leading the company’s government work said in a court filing this year that the CDC workers were using a general model rather than a specialized one for government use. Anthropic worked with the agency to show how best to work with the company’s AI tools, he said in the filing.

OpenAI created a program to give vetted organizations and researchers access to models with fewer automated safety refusals. But the process for getting approved can be complicated, slowing approvals.

Sam Altman, OpenAI’s chief executive, recently joined other tech CEOs in calling for Congress to require safeguards for companies ordering synthetic DNA and RNA. These highly specialized goods are typically sold to labs and scientists for research work or to develop certain vaccines. Some industry analysts say AI tools make it easier for shady buyers to use them to make deadly new pathogens to unleash on the world.

Lawmakers have proposed bipartisan legislation to increase AI oversight. “AI is a powerful engine of innovation, and I want to see it flourish, but not without accountability and not without human oversight,” Rep. Nathaniel Moran (R., Texas) said. He spoke in June while introducing a bill requiring AI companies to report evidence of dangerous threats, including those posed by biological weapons, to the Commerce Department.

Moran is co-sponsoring a separate bill giving the federal government authority to order tech companies to shut down AI models deemed too dangerous.

Make or break

By 2024, OpenAI’s biology skills were fast improving. Company tests showed how a biologist could persuade ChatGPT to give coaching on how to build aerosolized pathogens with enough user questions, according to people familiar with OpenAI’s development.

At the time, employees predicted that by 2025, ChatGPT would be able to help users who had taken only high-school biology to design and make a biological weapon.

One of OpenAI’s safety executives at the time, Ryan Beiermeister, argued with colleagues about the chatbot’s advancing skills. She said company safety employees needed to quickly figure out how to best detect users asking ChatGPT to help make biological weapons and plan attacks, according to people familiar with the conversations.

Some executives dismissed her concerns, saying the model’s existing safeguards were enough. Beiermeister nonetheless helped mobilize groups across the company’s safety teams, even ones she didn’t manage, to build a monitoring system.

By spring 2025, the team had a rudimentary product to identify users pursuing biological weapons. In a June 2025 blog post, the company acknowledged that advancing biology capabilities of its models could be misused and said it was working on detection and enforcement systems, training the model to refuse harmful requests.

That summer, as OpenAI prepared to release GPT-5, employees determined the chatbot had hit a high-risk mark, defined by the company as ChatGPT successfully aiding a user with limited training to create a biological hazard.

Some employees feared the team’s detection tool wasn’t comprehensive, according to people familiar with the concerns. The company said it continued to refine the tool. OpenAI has monitored 100% of user queries for its advanced models since April 2025, according to the spokeswoman.

She said GPT-5 launched after it was trained not to provide information that could cause harm. The company offers a $50,000 bounty for users who can show they evaded certain safeguards regarding biological weapons.

Early this year, OpenAI cut ties with Beiermeister on the grounds of sexual discrimination, which she has disputed .

Soon after OpenAI released the new model, employees found GPT-5 helping users who asked about making poisons and biological weapons. Later in the fall, OpenAI changed the designation of GPT-5, classifying it as less dangerous.

Write to Georgia Wells at georgia.wells@wsj.com and Amrith Ramkumar at amrith.ramkumar@wsj.com