Developing Guardrails for AI Biodesign Tools

Responsibly harnessing the power of AI in bioscience and biotechnology research and development

The integration of artificial intelligence (AI) with the life sciences offers tremendous potential benefits to society, but advances in AI biodesign tools also pose significant risks of misuse, with the potential for global consequences.

AI biodesign tools (BDTs) are technologies that enable the engineering of biological systems. These tools are trained on biological data and are developed to provide insights, predictions, and designs related to biological systems. BDTs have the potential to drive progress in the development of new therapeutics and are likely to have a significant impact across the broader bioeconomy, including in agriculture, health, and materials science. However, there are risks BDTs could be misused to design dangerous pathogens, and few safeguards exist to ensure that the benefits of these technologies can be realized safely and securely.

Innovative strategies are needed to reduce the risks associated with potential misuse of biological design tools without significantly hindering beneficial uses. This report identifies a number of strategies, referred to as guardrails, that could be developed to safeguard BDTs against misuse.

Focus Areas

The report identifies and makes recommendations across a range of guardrails that could be developed to help safeguard biological design tools, with a focus on two key areas:

• Built-in guardrails, which refer to technical solutions for risk reduction that can be included in the development or use of a BDT.
• Managed access paradigms, which includes ways to provide differential access to BDT software—through a range of approaches on the continuum between open-source and closed models—based on the needs of different developers and users.

This report draws on interviews with more than 20 experts in AI, biosecurity, policy, and biological design tool development to outline a set of strategies for securing the tremendous benefits of BDTs while reducing the risk that they could be exploited to cause harm. Following the interviews, NTI convened technical experts to discuss and solicit feedback to refine these recommendations.

Next Steps and Potential Pilot Projects

The report recommends several opportunities for the scientific community, biological design tool developers, and biosecurity experts to explore a range of proposed guardrails. The authors detail potential pilot projects that explore the implementation of built-in guardrails and managed access approaches:

Built-In Guardrails

• Developing an ecosystem to support screening and refusals. Screening mechanisms could be used to automatically detect and flag potentially risky BDT inputs or outputs, which could then be flagged again for further review or rejected. This method relies on biosecurity experts working alongside BDT developers, sharing insights from DNA synthesis screening, and testing this approach in a commercial context.

• Coupling designs with metadata. Capturing and cryptographically signing metadata created during the biological design process—for example, when using protein design tools—could be used to infer user intentions. This information could be shared with DNA synthesis providers and others to improve biosecurity screening.

• Curating biological training data. Excluding virus and toxin data from the training datasets might prevent BDTs from generating dangerous designs.

Managed Access 

• Establishing and supporting a managed access platform for BDTs. The platform would offer resources, ease of use, and collaboration features, while ensuring oversight and limiting access so only legitimate developers could use the BDTs.

• Gathering information on the users of BDTs. A better understanding of how users access BDTs and their reasons for choosing various access methods would provide valuable insights for improving managed access to these tools.

• Developing a written framework for managed access to BDTs. Building on existing guidelines for other AI models, this framework would address different levels of access, best practices, and ways that access might change over the course of the BDT’s development, with input from both academic and industry developers.
• Developing a written framework for managed access to newly generated data. Given that a significant amount of biological data is yet to be generated, a framework is needed to ensure proper data management as investments in data generation continue to grow.

The built-in guardrails and managed access paradigms identified and recommended in this report offer a set of approaches to secure the benefits of biological design tools while reducing the risk of their misuse by malicious actors. Exploring the feasibility and testing these strategies though a range of pilot projects can help reduce risks while supporting innovation.

Download the full report.