Anthropic partners with US government to prevent AI nuclear leaks

Anthropic partners with US government to prevent AI nuclear leaks
15 Nov 2024

In a first, leading artificial intelligence (AI) firm Anthropic has partnered with US Department of Energy's (DOE) nuclear experts.

The collaboration is focused on making sure that Anthropic's AI models do not accidentally leak sensitive information about nuclear weapons.

The initiative, which started in April and was revealed by Anthropic to Axios, is a major step in AI security.

DOE experts test Anthropic's AI model for potential misuse
Security assessment

The DOE's National Nuclear Security Administration (NNSA) is conducting a "red-teaming" exercise on Anthropic's AI model, Claude 3 Sonnet.

The process involves experts trying to exploit or break a system to find potential weaknesses.

The main focus of this exercise is to see if Claude's responses could be manipulated for creating nuclear weapons or accessing other harmful nuclear applications.

Anthropic's AI model undergoes rigorous security tests
Ongoing evaluation

The security assessment project will continue until February, during which the NNSA will also assess the updated Claude 3.5 Sonnet introduced in June.

To prepare Claude for these intensive government-focused security tests, Anthropic has leveraged its partnership with Amazon Web Services (AWS).

However, given the sensitive nature of this work, Anthropic has not yet revealed any findings from this pilot program.

Plans to share security assessment results
Information sharing

Anthropic intends to share the findings of its security assessment with scientific labs and other organizations, encouraging independent testing to avoid the potential misuse of AI models.

Marina Favaro, Anthropic's national security policy lead, stressed the need for partnerships between tech companies and federal agencies in evaluating national security risks.

AI at the heart of national security discussions
Security focus

Wendin Smith from the NNSA, emphasized that AI has become a focal point in important national security conversations.

She said that their agency is well-prepared to assess possible risks posed by AI, especially those pertaining to nuclear and radiological safety.

This partnership comes in line with President Joe Biden's recent national security memo calling for AI safety assessments in classified settings.

Developers seek government contracts amid safety concerns

As AI developers battle for government contracts, Anthropic recently teamed up with Palantir and AWS to provide Claude to US intelligence agencies.

Other giants like OpenAI have also joined forces with organizations like NASA and the Treasury Department.

However, it remains unclear how these partnerships will hold up in the face of looming political changes in Washington.