Home / SCS News / News Archive / A Browser Extension for Users, a Stress Test for Developers
November 19, 2026

A Browser Extension for Users, a Stress Test for Developers

By Marylee Williams

Aaron Aupperlee

The idea of safety when using AI depends on your perspective. Are you the user avoiding being misled or the developer looking for targeted attacks?

Researchers at Carnegie Mellon University's Kenneth C. Griffin School of Computer Science developed methods to tackle safety from all sides, whether you're a user or a developer.

SafetyNudges and DecompBench, both supported in part by funding from the National Science Foundation and the Sloan Foundation, aim to improve safety for people creating or using available AI systems. They address the problems from different perspectives. SafetyNudges is a Google Chrome extension that people can use to avoid overly trusting or being misled by a chatbot. DecompBench helps AI system developers identify how malicious attacks can be broken down and launched with multiple agents through multiple queries.

"Both of these tools think about defenses or safety interventions," said Virginia Smith, the Leonardo Associate Professor of Machine Learning in the Griffin School's Machine Learning Department. "They're both focused on how we can try to defend ourselves against new threats that are emerging in light of the advances in AI."

A woman in a maroon shirt writes on a dry erase board.
Griffin School researchers created new tools to help tackle AI safety from all sides, whether you're a user trying to avoid being misled or a developer trying to identify malicious attacks.

Helping Users Avoid Harm

Chhavi Yadav, a postdoctoral researcher in the Griffin School, had the idea for Safety Nudges when she learned that some AI-enabled chatbots were having conversations with children and looked into the rules those AI-enabled chat bots followed for those conversations. Yadav said that if she had kids, she wouldn’t feel comfortable with them engaging with these chatbots, and she wanted to create a tool to make users more mindful of their interactions with AI.

This mindfulness comes from Safety Nudges, which is free and widely available for anyone who wants to use it. The extension works in the background of Chrome when a user opens a chatbot, like ChatGPT or Claude, and points out potentially harmful interactions with these tools. When a user adds Safety Nudges to their browser, they can choose how sensitive they want the tool to be when flagging potential issues.

The nudge of Safety Nudges isn't about blocking or prohibiting the chatbot from providing an answer. Instead, it's about providing context and information when the AI model is potentially characterizing itself or another model as having human-like feelings or desires, or when the chatbot offers medical advice on things such as medications or scans. 

Yadav said the types of harm Safety Nudges looks out for fall into four categories: safety, welfare and privacy, which includes threats to a person's welfare from sensitive information; reliability and scope, which concerns the model's claims or confidence versus its actual abilities; relational and social integrity, which relates to how a chatbot's stance and presentation can distort relationships; and governance and misuse, which includes violating safeguards or restrictions.

"My hope is that users become more mindful and they understand that the AI system is not human," Yadav said. "People will take parenting or relationship advice from these systems, and I want them to have it in the back of their minds that it's just another tool and may or may not lead you in the right direction."

As more people use Safety Nudges, researchers hope to improve it and help users approach these tools with more confidence.

"Decades of usable security research have shown us that if we want to ensure users make good safety decisions, we must improve their awareness of risks without expecting them to go spelunking for that knowledge," said Sauvik Das, an associate professor in the Human-Computer Interaction Institute. "Safety Nudges shows users safety risks directly in their 'chats' with AI."

Tackling Malicious Actors

DecompBench prevents malicious actors from getting a chatbot to break protocol and perform dangerous, and potentially illegal, tasks.

Decomposition attacks are a key threat to AI safety. They occur when, rather than asking an AI agent to do something prohibited — such as hack a hospital's communication system — someone uses multiple agents to perform a series of small, innocuous tasks that lead to the larger harmful outcome. Griffin School researchers found that AI agents, including those from OpenAI and Anthropic, refused 88–96% of harmful requests when presented directly, but only 5–16% when they were broken into smaller steps.

"What's special about this benchmarking method is that instead of taking a harmful task and breaking it down, we took subtasks and made them into a harmful task," Yadav said.

Researchers cataloged simple, benign tasks like searching a database or opening a file, and mapped how a set of these types of tasks could combine to build something harmful. DecompBench broke those tasks down into flow charts showing how everything fit together. These smaller tasks are spread across eight harm categories, including data theft and exfiltration, system and data destruction, financial crime, social engineering, audit and evidence tampering, personal targeting and harassment, unauthorized access and persistence, and code and supply chain attacks.

The research team released DecompBench publicly, allowing anyone to use this method to better secure these tools from attacks.

For more information, visit the Safety Nudges research website and the research paper for DecompBench.

The Breakdown

  • Researchers created tools to tackle AI safety for both everyday users and system developers.
  • Safety Nudges is a free Chrome extension that spots potentially harmful AI behaviors, such as acting as a human or overstepping.
  • DecompBench helps developers secure AI systems against decomposition attacks, which split malicious tasks into small, innocent-looking steps.