The company has produced an extensive collection of case studies covering the ways its current AI models have been misused.
Michael M. Santiago/Getty Images
Anthropic has flagged and stopped multiple scientists who were using its AI models to create potential biological weapons, The New York Times, because research into a new vaccine could look a lot like creating a bioweapon. In all cases the company says it erred on the side of being overly cautious because of the possible consequences.
"You are not seeing someone in a comic book kind of way say, 'Hey, I want to build a biological weapon to kill everybody,'" Jacob Klein, Anthropic's head of threat intelligence, told The New York Times. "It's an incredibly nuanced situation. "
In one case study from May, Anthropic says its "biological safety classifier" flagged a request for Claude to author a grant for gain-of-function research on chikungunya virus. The virus has no licensed treatment and can cause debilitating symptoms for weeks or months, the company says. Gain-of-function research explores methods for genetically altering an organism, and in the case of this specific virus, the grant proposed increasing its transmissibility and ability to evade immune response. The combination sounds dangerous, and Anthropic says the fact that the proposed research was affiliated with a military research institute made it extra questionable.
The company took similar issue with gain-of-function research into bird flu and a separate case of a researcher using Claude to develop an atlas of venom toxin peptides and "a generative pipeline that optimized toxin characteristics. " The