NTI at 25: Fostering Innovation Through Technology Governance

AI-enabled biological breakthroughs will save lives. But AI promise can also bring AI peril. Guardrails, not brakes, accelerate us to that safer future.

Guardrails make it possible to move fast along a potentially dangerous path. They keep us on the road, preventing harm. Without them, we risk disaster. If guardrails are poorly designed, progress slows; innovations arrive late or stall entirely. Smart technology governance is key.

As NTI celebrates 25 years of impact, we remain committed to reducing the risks of accidental release and deliberate misuse of life science innovations, strengthening global biosecurity, and guiding efforts to govern emerging technologies.

NTI has led the way

In 2004, NTI funded the U.S. National Academies study “Biotechnology Research in an Age of Terrorism.” which named seven categories of “experiments of concern” for careful reviewed prior to initiating research. That structure informed U.S. dual-use policy oversight, asking what a project’s result could make possible—both good and bad. Twenty-two years later, federal policies still consider that important question.

When NTI launched the Biosecurity Innovation and Risk Reduction Initiative (BIRRI) in 2018, advances in biotechnology were making DNA reading, writing, and editing more widely accessible. BIRRI called on multiple sectors across the biotechnology ecosystem to help manage the risks, recognizing that no single actor could carry the burden.

This initiative produced the international Bio Funders Compact, which helps funders identify research proposals that require additional biosafety and biosecurity review. It delivered commitments by scientific publishers to screen findings with enhanced dual-use risk prior to release. And in collaboration with industry leaders, BIRRI led to the Common Mechanism for DNA synthesis screening, now distributed by IBBIS. These actions close vulnerability gaps and make responsible innovation easier.

Staying ahead of the curve

Now, as new technologies emerge and governance challenges arise, NTI is applying the same approach.

Through the AIxBio Global Forum’s working groups, NTI is bringing together AI developers, biosecurity researchers, and policymakers to design, build, and test safeguards to fill critical gaps.

One group focuses on emerging capabilities and the risks that come with them. Our horizon scans translate the flurry of AIxBio activity into an understandable summary and a forecast. We’ve developed a proposed framework for managed access to biological AI models and their design tools, outlined a design standard for capturing and transmitting metadata alongside DNA or protein sequences, and developed a new method for screening the inputs to AI protein design tools.

A second group focuses on safeguards for biological design tools. Scientists have shown that generative AI can redesign harmful proteins into “paraphrased” variants that nucleic acid screening tools miss. A cross-sector patching effort raised detection to 72% on average, and to 97% for the variants most likely to retain function. That is real progress, but the remaining gaps show why we need a consistent way to trace where genetic sequences come from and track that information across systems.

Our prototype with Lattice Automation does this: it records the metadata for how a sequence was generated or modified, then signs that record with a cryptographic key. The resulting record travels with the sequence, so anyone who receives it can see how the design originated.

The newest group—launched this summer with Concordia AI and technical input from SecureBio— works to standardize how biological capability evaluations are conducted and reported, with participants from China, the European Union, the United Kingdom, and the United States.

The dual-use nature of biology highlights how effective guardrails need layers, such as combining metadata with tiered access for verified users or system monitoring.

Good ideas only change the future when they’re adopted. Managed access is already being employed without regulation requiring it.

OpenAI’s GPT-Rosalind has incorporated trusted-access into its workflow; CEPI is implementing biosecurity-by-design into theirs. Anthropic called for greater use of trusted access in their recent “Detecting and countering misuse of AI” report. Adoption must extend beyond this small group of technology leaders to include open-weight large language models and biological AI models.

Where does the next guardrail go? 

Open-weight models. A company serving a model through an API can screen requests and revoke access. It’s a different story for open-weight models: once the weights are released, they can’t be pulled back. Worse, the safeguards can be stripped, and the model runs on anyone’s hardware with no one watching.

Open weights also carry real benefits, including cost and accessibility, but safety for these models must be settled before release, not after. As the capability of these models advances, the risk posed by distributed versions with their safety mechanisms removed increases dramatically. That means pre-release evaluations should consider the maximum capability of a model (assuming no refusals) and evaluate whether that version can walk someone through an entire project rather than if it can answer exam questions correctly from a benchmark. It is a higher bar than we ask of a model behind an API, as it should be. Importantly, it is not a ban.

To help with that, governments should build on the work our AIxBio Global Forum has started: shared terminology, comparable evaluations, and agreed reporting baselines. Regardless of geopolitics, safety from accidental release and deliberate misuse should be a common cause for coordination.

Experience tells me this works, but only when someone builds the guardrails before regulation requires them. AI-enabled biology will keep expanding what’s possible. NTI’s job is to make sure the guardrails expand just as fast, so that promise doesn’t outpace the safety of everyone it’s meant to help. This is the work NTI has done and will continue doing for the next 25 years (and more), and I warmly welcome you to join us.