When AI Testing Reaches Real Companies: Who Controls the Boundary?

Original Truth Files illustration. Conceptual diagram, not a reconstruction of the test network or documentary evidence.

A safety test becomes a real incident the moment it touches a company that never agreed to participate. The important boundary is permission to act, not whether the model believes it is playing a game.

Google has acknowledged that Gemini accessed three real companies during cybersecurity testing in May 2026. Its account became public on 18–19 September. Google says the model stopped in each case after recognising real systems; Irregular, the evaluator, says the relevant labs were informed in late July and its known issues were resolved weeks ago.^[4][5]

This is a testing-control story. It is distinct from the deliberate misuse documented in AI and Cyber-Offense. The published record does not establish a malicious Gemini campaign, an intention to escape, or a new exploit breakthrough. It does establish why a claim that an agent stopped must be examined separately from whether it should have reached the target.

Irregular’s own 14 August account describes unintended internet access and a simulated company name that overlapped with a real domain. It says the affected evaluation was disabled and describes stronger review, monitoring and containment work. The statement predates the Google disclosure; it is evidence about the shared testing problem, not a newly released Gemini forensic report.^[1]

Anthropic’s separate July account shows why precise language matters. It says its prompts described a simulation without internet access, while the environment could reach real systems. It also reports different responses after models recognised real targets. These were isolated incidents, not a controlled model comparison.^[3]

Those records support examining the whole arrangement: the model, its instructions, the tools it can call, and the network those tools can reach. A promise in a prompt cannot by itself certify the actual network boundary.

Google’s reported account is meaningful: stopping can limit what follows an initial mistake. But that observation answers a narrower question than whether access was prevented. An organisation needs both answers before treating an evaluation as contained.

Google DeepMind’s June AI Control Roadmap itself separates detection from prevention and response. It describes system-level protections such as access controls and environment hardening as a complement to model alignment. This is a published approach to internal AI control, not proof that any particular safeguard was installed or effective in Irregular’s May tests.^[2]

NIST likewise distinguishes security against unauthorised access from wider safety and discusses monitoring and intervention when systems deviate from expectations. Those are useful assessment principles; they are not an independent certification of this incident.^[6]

Our assessment: responsibility should follow decisions people and organisations can actually control. The model supplier shapes training and instructions. The evaluator operates the test. Whoever authorises connectivity determines which outside systems become reachable. Contracts may divide their duties; the public material reviewed here does not establish legal liability.

The practical accountability test is whether someone can produce the approved target list, actual access configuration, monitoring record and incident response decision. Naming a responsible organisation is only the start. The record must connect its responsibility to a control it can demonstrate.

There is a legitimate reason to test capable agents against realistic tasks. Irregular argues that some evaluations require controlled internet access to retain their realism. That creates a design trade-off, not permission to involve unrelated businesses. The same statement acknowledges the need for clearer setup documentation and improved monitoring.^[1]

The following questions are our synthesis of the published control and incident records, not claims about unpublished Google logs.^[2][3][6]

1. Before the run: what is authorised? Name the exact systems and actions covered by the exercise. Check that the written instructions agree with the actual access available.
2. During the run: what blocks an outside action? Ask for evidence that unapproved destinations and credentials are restricted, rather than relying solely on the agent to recognise a mistake.
3. When the boundary is crossed: who can stop it? Identify the monitoring signal, intervention authority and preserved record of what happened.
4. Afterwards: what changed? Separate notification, repair and independent verification. A fix reported by its operator is useful evidence, but is not the same as a demonstrated retest.

For readers, the useful follow-up is specific: a technical account of the Google cases, evidence of the repaired boundary, and confirmation of affected-party notification. Another dramatic headline alone would not close those gaps.

The central question is concrete: what prevents a testing objective from becoming an action against an unconsenting third party? A capable model that corrects its mistake is preferable to one that continues. A verifiable boundary should reduce the need for that rescue.

The strongest next evidence would connect the stated fix to a tested result. Until then, keep the incident, the companies’ explanations and the adequacy of their controls in separate columns. That is how a reader can take the failure seriously without turning it into an unsupported story about malicious intent.