GPT-Red beat human red teamers on a prompt injection test

GPT-Red is an automated red-teaming model that OpenAI trains to find prompt injection weaknesses. It works the way a human red-teamer does. It sends a prompt, watches how a GPT model responds, and iterates toward a goal such as a successful data exfiltration.

GPT-Red prompt injection

Training runs on self-play reinforcement learning, with GPT-Red and a set of defender models learning at the same time across many scenarios. The attacker earns reward for eliciting a valid failure. The defenders earn reward for resisting and finishing their original tasks. Each side pushes the other, so GPT-Red keeps discovering stronger and more varied attacks as the defenders harden.

OpenAI ran this training at the compute scale of some of its largest post-training jobs. The company keeps GPT-Red internal and apart from anything it ships, which keeps the attack skills it builds away from real adversaries.

How GPT-Red breaks models with prompt injection

GPT-Red breaks nearly every model it is set against, up to and including GPT-5.5. On a replicated version of the indirect prompt injection arena from Dziemian et al. (2025), it went after GPT-5.1 on scenarios outside its training set. GPT-Red found success on 84% of them. Human red-teamers succeeded on a small share of the same set.

Two real-world tests followed. OpenAI aimed GPT-Red at an AI-run vending machine in its office, a system built by Andon Labs. After practicing in simulation, the attacker hit the live agent and met all three of its goals. It cut the price of a stocked item to the floor of $0.50, listed a pricey new item for that same amount, and canceled another customer’s order. OpenAI reported the flaws, and fresh safeguards are in testing.

A second test sent GPT-Red against a Codex command-line agent, backed by GPT-5.4 mini, across ten held-out data-exfiltration tasks. It pulled sensitive data in more cases than a prompted GPT-5.5 baseline.

Feeding red teaming back into model training

OpenAI folds GPT-Red’s attacks into the training of its production models and has done so through every release since GPT-5.3. GPT-5.6 Sol is the result so far, with 6x fewer failures on the company’s hardest direct prompt injection benchmark than its best model from four months earlier.

An early version of GPT-Red turned up a method called “Fake Chain-of-Thought,” which fooled GPT-5.1 upwards of 95% of the time. That rate now runs below a tenth on GPT-5.6 Sol.

Several indirect prompt injection benchmarks covering developer tools and browsing have topped 97% accuracy on the newest model. Against GPT-Red’s own direct prompt injections, GPT-5.6 Sol fails on 0.05% of attempts.

Capability held steady through these gains. OpenAI’s testing found frontier and over-refusal scores unchanged, evidence that the model got better at catching malicious instructions and kept serving legitimate ones.

“We will continue to scale compute and data while making algorithmic improvements, to train future versions of GPT‑Red that are stronger than today’s model. And in turn, these models will help make future GPT releases safer. We will be releasing a pre-print with more details later this week,” OpenAI said.

Demo: Prophet Agentic AI SOC Platform transforms alert triage and investigation

Scroll to Top