Risky Business

People Are Trying to Misuse AI. That’s Exactly Why Safeguards Matter.

One of the challenges in the AI risk debate is that discussions often begin and end with hypotheticals. In conversations about AI and biological risks, you’ll often find two camps: those who see little reason for concern and those who view the risk as existential. Too often, the debate becomes less about evidence and more about competing assumptions.

That is why Anthropic’s latest threat intelligence report is so important. Most experts already concluded that as AI capabilities continue to advance, they would be likely to lower barriers to malicious actors causing harm with biology. Rather than asking what someone might do with advanced AI systems, the report documents the dangerous activities people have already tried to do.

Importantly, this is not a story about AI systems running amok. The most encouraging part of the report is that the examples of attempted misuse were detected, disrupted, and used to improve safeguards. Anthropic reports that in each case it disrupted the activity, strengthened its defenses, and shared intelligence with relevant authorities and industry partners when appropriate.

For those of us who believe AI can deliver enormous benefits across health, economic prosperity, and national security, that distinction matters. The emergence of misuse does not negate the value of the technology. Every transformative technology presents both opportunities and risks. The real question is whether we can responsibly manage those risks while preserving the benefits.

The report provides evidence that people are actively testing the boundaries of these systems. In the biological domain, Anthropic describes multiple cases in which researchers attempted to use its models for research with potential dual-use implications. As the report notes, “Biological capabilities are dual use: they can be used for beneficial or harmful purposes, and it is often difficult to distinguish between them.”

That observation gets at the heart of the challenge. Most advanced biological knowledge has legitimate applications. The same information that can support vaccine development or improve public health can, under different circumstances, be misused. The existence of dual-use risks does not mean the research should stop. It does mean that safeguards, monitoring, and responsible deployment become essential.

What I appreciate most about this report is its transparency. According to Anthropic, these are “examples of the most notable and novel threat activity we’ve identified to date,” and the company notes that it is publishing them because it believes it has “a responsibility to disclose malicious misuse of our services.”

Public discussions about AI risk are often hampered by a lack of real-world evidence. Here, we have a major AI developer showing its work: documenting attempted misuse, explaining how safeguards functioned, and identifying areas where defenses need to improve.

We should neither dismiss these findings nor overreact to them.

While this report does not respond to recent suggestions that an AI-driven catastrophe is around the corner, it does demonstrate something concrete and actionable: people are already seeking to use advanced AI systems for opaque, risky, and potentially harmful purposes. The good news is that the risks described remain manageable precisely because companies, researchers, and governments are investing in safeguards, detection capabilities, and transparency.

That is the lesson I took away from this report. The debate is no longer about whether anyone would try to misuse AI. We now have clear evidence that some actors will.

The policy challenge ahead is ensuring that safeguards, governance frameworks, and security practices continue to responsibly scale alongside the technology. If they do, society can capture AI’s enormous benefits while keeping emerging risks firmly under control.

Stay Informed

Sign up for our newsletter to get the latest on nuclear and biological threats.

Sign Up

Congress Must Act to Secure U.S. Biotechnology

Risky Business

Congress Must Act to Secure U.S. Biotechnology

Innovation requires security, and security requires innovation. Congress needs to act decisively to ensure U.S. leadership in biotechnology is paired with governance that keeps its development secure.


Improving Biosecurity through Metadata Capture

Risky Business

Improving Biosecurity through Metadata Capture

As the capabilities of biological AI tools continue to advance at an accelerating pace, it is vital that DNA synthesis providers and others in the biosecurity space embrace new guardrails to prevent their misuse.



See All

Close

My Resources

Subscribe to NTI

Sign up for regular updates on innovative, real-world solutions to existential threats.

Get Updates