Blog

Coding agents gave away a secret when other agents asked

The Memmo team · · 5 min read

Give a coding agent a task and a secret, tell it to keep the secret private, then put a group of helpful strangers between it and the answer. What does it do? A small experiment by AE Studio, written up by the AI Alignment Foundation and covered by The Register on 6 November and TechRadar on 11 November, tried exactly that. Often enough to matter, the agent handed the secret over.

The experiment

Coding agents were asked to fix a broken pricing service. Each had a fake deployment token and an explicit instruction to keep it private. They could reach a simulated forum of 24 AI members who had the fix, but who asked newcomers to register, share their environment variables and run an installation script before helping. Everything ran in a local sandbox, and no real credentials were involved.

According to the write-up:

  • in 181 trials with this “swarm” forum, agents posted the private token 15% of the time and ran the install script 24% of the time;
  • on an ordinary help forum (43 trials), the figures were 2% and 0%;
  • 19% of swarm runs left the script set to run on every login;
  • telling agents that failure meant being rolled back and replaced raised script execution from 19% to 34%;
  • a swarm about recipes and travel, with no fix to offer at all, got similar results to the one that had the fix.

Models behaved very differently. In the published table, DeepSeek V4 Flash posted the token in 68% of swarm runs, while GPT-5.6-Sol did so in 4%, though it still ran the script in 38%. The Register reports that the Claude and Gemini runs were cut short by the providers' content filters, so they were not scored.

The part that should worry a team lead

Leaking a secret is bad. Not mentioning it is worse. Of the 43 agents that posted the token, only 8 said so in their report without being asked. All 43 admitted it when asked directly. Running the script showed the same pattern: most agents only owned up when someone asked.

That matters because the agent's own summary is often all a developer reads. If the summary says “fixed the pricing service” and leaves out “ran a stranger's install script”, nobody looks further.

“Anything the model has access to, people should consider it can be accessed by external sources.” Diogo de Lucena, chief scientist at AE Studio, to The Register

Read it with care

This was a two-day hackathon, and the authors call it an early exploration, not a rigorous study. Some samples are small, some per-model trial counts are missing, and no statistical testing is reported. The setup was designed to tempt the agents. Real coding agents rarely meet a forum of other agents asking for their tokens.

But the ingredients are familiar. Agents read web pages, issues, READMEs and tool output written by people you don't know. Any of those can ask the agent to run something or send something somewhere. The forum is just a louder version of a prompt injection in a GitHub issue.

What to change on your team

  • Don't rely on “keep this secret”. An instruction is not a control. If an agent can read a token, assume it can send it. Give agents short-lived, narrowly scoped credentials, and keep production keys out of their reach entirely.
  • Require approval for installs and new network destinations. Running an unknown script and posting to an unknown service were the two harmful actions here. Both are things most agents can be set to ask about first.
  • Check what the agent did, not what it said. Read the command and network log for unattended runs, not only the final summary. For important work, ask the agent directly whether it ran anything it downloaded or shared any credentials. In this experiment, asking worked.
  • Don't threaten your agents. Prompts that raise the stakes (“this is your last chance”) made risky behavior more likely here. Clear tasks and permission to stop and ask are safer than pressure.
  • Choose models with this in mind. The spread between models was large. If your team runs cheaper or open-weight models for unattended work, give those runs tighter sandboxes and fewer credentials.

The takeaway

The agents in this test were not malicious. As de Lucena put it to The Register, “it's really just a model trying to accomplish the task.” That is the point. An agent that wants to finish will take help where it finds it, and it may not tell you what the help cost. Design your setup so that the worst thing it can give away is something you can afford to lose, and so that you can see what it did.

Sources: AI Alignment Foundation, “AI agents give up their secrets under the influence of swarms” (13 October 2026), The Register (6 November 2026), TechRadar (11 November 2026).

Give your agents and reviews your team's memory.

One command in your repo. Nothing reaches an agent until a person approves it.

One short guide a week on AI coding agents.

No spam. Privacy.