NTI | bio and Concordia AI, with technical input from SecureBio, have launched the AIxBio Technical Working Group on Evaluation Practice. Convened through the AIxBio Global Forum, the Working Group will bring together experts from organizations in China, the European Union, the United Kingdom, and the United States that develop or conduct biological capability evaluations of AI systems.
Biological capability evaluations increasingly inform frontier AI risk management by assessing capabilities that could lower barriers to biological misuse. Model developers use the findings when deciding whether to strengthen safeguards, restrict access, increase monitoring, or change deployment plans.
As these evaluations become more widely used, findings produced in one setting may inform risk assessments and related decisions in another, even when evaluation methods and reporting practices differ. Results reflect not only the capabilities of the model being evaluated, but how those capabilities are tested. Without consistent reporting, differences arising from system configuration or evaluation design may be mistakenly attributed to the underlying model itself.
The Working Group will compare published evaluation approaches across jurisdictions and institutional settings to establish a common technical basis for designing evaluations and interpreting their results. Building on this analysis, it will develop shared terminology and a minimum reporting baseline for evaluation results. Its work will:
Methodological findings, voluntary reporting conventions, and research priorities will be published to support more consistent interpretation and use of evaluation evidence by AI developers, governments, and third-party evaluators.
By bringing together experts who conduct this work to examine assessment design and comparability, the group will complement the AIxBio Forum’s Working Group on Horizon Scanning, Risk Assessment, and Evaluations.
Organizations and individuals working on biological capability evaluations are invited to contact Helia Samani at [email protected] to discuss participation or share related work.
Sign up for our newsletter to get the latest on nuclear and biological threats.
Amid rapid advances in bioscience and as artificial intelligence reshapes global technological capabilities, the Nuclear Threat Initiative (NTI) and the China Arms Control and Disarmament Association (CACDA) are jointly calling for action to strengthen biosecurity and oversight and engage in responsible practices to prevent accidents or misuse.
New NTI | bio paper calls on tool developers and other AIxBio stakeholders to implement a managed access approach—allowing only validated users to access those biological AI models that contribute to biosecurity risks—to help reduce the risk that such tools will be deliberately misused to cause harm.
NTI | bio’s Dr. Aparupa Sengupta joined a groundbreaking discussion on the nexus of cyber- and bio-security.
Sign up for regular updates on innovative, real-world solutions to existential threats.
