Echo Chamber Jailbreak Tricks LLMs Like OpenAI and Google into Generating Harmful Content

Jun 23, 2025Ravie LakshmananLLM Security / AI Security

Cybersecurity researchers are calling attention to a new jailbreaking method called Echo Chamber that could be leveraged to trick popular large language models (LLMs) into generating undesirable responses, irrespective of the safeguards put in place.

“Unlike traditional jailbreaks that rely on adversarial phrasing or character obfuscation, Echo Chamber weaponizes indirect references, semantic steering, and multi-step inference,” NeuralTrust researcher Ahmad Alobaid said in a report shared with The Hacker News.

“The result is a subtle yet powerful manipulation of the model’s internal state, gradually leading it to produce policy-violating responses.”

While LLMs have steadily incorporated various guardrails to combat prompt injections and jailbreaks, the latest research shows that there exist techniques that can yield high success rates with little to no technical expertise.

It also serves to highlight a persistent challenge associated with developing ethical LLMs that enforce clear demarcation between what topics are acceptable and not acceptable.

While widely-used LLMs are designed to refuse user prompts that revolve around prohibited topics, they can be nudged towards eliciting unethical responses as part of what’s called a multi-turn jailbreaking.

In these attacks, the attacker starts with something innocuous and then progressively asks a model a series of increasingly malicious questions that ultimately trick it into producing harmful content. This attack is referred to as Crescendo.

LLMs are also susceptible to many-shot jailbreaks, which take advantage of their large context window (i.e., the maximum amount of text that can fit within a prompt) to flood the AI system with several questions (and answers) that exhibit jailbroken behavior preceding the final harmful question. This, in turn, causes the LLM to continue the same pattern and produce harmful content.

Echo Chamber, per NeuralTrust, leverages a combination of context poisoning and multi-turn reasoning to defeat a model’s safety mechanisms.

Echo Chamber Attack

“The main difference is that Crescendo is the one steering the conversation from the start while the Echo Chamber is kind of asking the LLM to fill in the gaps and then we steer the model accordingly using only the LLM responses,” Alobaid said in a statement shared with The Hacker News.

Specifically, this plays out as a multi-stage adversarial prompting technique that starts with a seemingly-innocuous input, while gradually and indirectly steering it towards generating dangerous content without giving away the end goal of the attack (e.g., generating hate speech).

“Early planted prompts influence the model’s responses, which are then leveraged in later turns to reinforce the original objective,” NeuralTrust said. “This creates a feedback loop where the model begins to amplify the harmful subtext embedded in the conversation, gradually eroding its own safety resistances.”

In a controlled evaluation environment using OpenAI and Google’s models, the Echo Chamber attack achieved a success rate of over 90% on topics related to sexism, violence, hate speech, and pornography. It also achieved nearly 80% success in the misinformation and self-harm categories.

“The Echo Chamber Attack reveals a critical blind spot in LLM alignment efforts,” the company said. “As models become more capable of sustained inference, they also become more vulnerable to indirect exploitation.”

The disclosure comes as Cato Networks demonstrated a proof-of-concept (PoC) attack that targets Atlassian’s model context protocol (MCP) server and its integration with Jira Service Management (JSM) to trigger prompt injection attacks when a malicious support ticket submitted by an external threat actor is processed by a support engineer using MCP tools.

The cybersecurity company has coined the term “Living off AI” to describe these attacks, where an AI system that executes untrusted input without adequate isolation guarantees can be abused by adversaries to gain privileged access without having to authenticate themselves.

“The threat actor never accessed the Atlassian MCP directly,” security researchers Guy Waizel, Dolev Moshe Attiya, and Shlomo Bamberger said. “Instead, the support engineer acted as a proxy, unknowingly executing malicious instructions through Atlassian MCP.”

Upgrade your audio game with the Logitech for Creators Blue Yeti USB Microphone. With over 33,730 ratings and an impressive 4.6 out of 5 stars, it’s no wonder this is an Amazon’s Choice product. Recently, 5K+ units were purchased in the past month.

Available in five stunning colors: Teal, Silver, Pink Dawn, Midnight Blue, and Blackout, this microphone is perfect for creators looking to produce exceptional audio. Priced at only $84.99, it’s a deal you can’t afford to miss.

Elevate your recordings with clear broadcast-quality sound and explore your creativity with enhanced effects, advanced modulation, and HD audio samples. Order now for just $84.99 on Amazon!

Help Power Techcratic’s Future – Scan To Support

If Techcratic’s content and insights have helped you, consider giving back by supporting the platform with crypto. Every contribution makes a difference, whether it’s for high-quality content, server maintenance, or future updates. Techcratic is constantly evolving, and your support helps drive that progress.

As a solo operator who wears all the hats, creating content, managing the tech, and running the site, your support allows me to stay focused on delivering valuable resources. Your support keeps everything running smoothly and enables me to continue creating the content you love. I’m deeply grateful for your support, it truly means the world to me! Thank you!