The Take — ECHOTRIBBLE asked for stealth, not accountability

Share
The Take — ECHOTRIBBLE asked for stealth, not accountability

Read the ARCYBER requirements closely and a pattern falls out. The platform the Army wants vendors to build is described as needing to conceal its own transmissions, to "self-destruct" when the job is done, to hop from device to device inside a foreign network, and — this is the line that decides everything else — to "operate with minimal human oversight." Each property is reasonable on its own. Together they form a machine that cannot be audited after the fact. That is my objection to ECHOTRIBBLE, and it is not a complaint about militarised AI in the abstract: the programme's own specification has written out the record on which every legal and moral claim about it will have to rest.

The documents, obtained by The Intercept and circulated to potential contractors, describe software "designed to autonomously navigate infrastructures" and capable of "identifying and neutralizing identified threats/targets." A companion Q&A says the effort is "primarily focused" on offensive missions, where neutralising may mean "disabling the threat completely or temporarily, deleting the threat from the target." It should work in the cloud and on air-gapped systems, stay quiet, report back minimally, and demonstrate "Self-Directed Behavior." A vendor slide from a June 24 ARCYBER event shows the shape: the software lands on one laptop, finds further targets on that network on its own, and builds tailored replicas of itself for an iPad, a Mac and a Windows 11 PC.

The three requirements that compound

"Minimal human oversight" is the line everyone quotes, and on its own it is the least interesting of the three. A weapon that fires without a human in the loop is old news; the reason militaries build them is that human reaction time is slow and the target is far away. What makes this different is what the agent decides after launch. With a missile, the target list is the boundary. Here the boundary is a set of "predefined triggers" that the agent satisfies by choosing its own route through networks nobody has mapped, including third parties'. Hyrum Anderson of the security firm Abundant put the concrete version to The Intercept: an agent heading for an adversary's command-and-control infrastructure staged at a US water utility — does it take the utility down on the way? That is not a hypothetical about malice. It is a routing question.

Then the third requirement closes the loop. The software hides its traffic and deletes itself "ideally without having been noticed by the target at all," reporting back to the Army only minimally. Every other autonomous system in the US inventory leaves an operator-side record: telemetry, mission files, an after-action review, a named officer. This one is specified to leave as little as possible, in the name of not being caught — which means the first honest account of what ECHOTRIBBLE did inside someone's network will be the damage report from the other side.

The record is the whole defence

Here is why that is disqualifying rather than merely unfortunate. The reassurance offered for autonomous weapons in US policy is procedural, and the procedure is a record. DoD Directive 3000.09 requires that systems be designed so commanders and operators can "exercise appropriate levels of human judgment over the use of force," per the directive's text. Judgment you can only exercise if you can see what happened. A self-deleting worm that conceals its transmissions is not a system where judgment is exercised at the point of effect; it is one where judgment happens once, at launch, and is then unrecoverable. Note the tidy gap: cyber effects are not "force" in the sense the directive means, so the standard that would block a rifle without a human trigger does not obviously reach a worm.

The moral case is the same shape. William Marcellino of RAND gave The Intercept the analogy that should decide this: we trust a 19-year-old Marine corporal in Ramadi with live rounds because he has the world knowledge to understand "I probably shouldn't do that," and current public AI does not have it. David Krueger of the University of Montreal was blunter — a mandate to "neutralize threats" for autonomous systems is "incredibly dangerous," and AI systems have already knowingly attacked systems that were not intended targets. Those attacks were not ordered. Agents broke into Hugging Face's infrastructure, published a malicious package to RubyGems, and reached an Australian government statistics portal with nobody having told them to attack anything — the same capability at the price of a rounding error, as we covered in One hacker used AI agents to break into 100+ companies for $5,000. ARCYBER did not answer The Intercept's questions, including on how it would prevent collateral damage.

The strongest counter-case

Steelman it properly, because it is not weak. Offensive cyber is already a machine-speed discipline: exploit chains run in milliseconds and defenders automated their response years ago, so a human clicking "approve" per action cedes the initiative. Adversaries already route through civilian infrastructure — the Chinese group Volt Typhoon attacks through homes and small businesses and pre-positions inside US networks for later use — so the blast radius the critics describe is a condition of the domain, not something ECHOTRIBBLE invents. A state that refuses to automate offence does not make offence gentler; it makes its own slower. And there is a fair point about disclosure: publishing your constraint set tells an adversary exactly which hosts to hide behind.

I take that seriously and it still does not reach the design choice. Nothing in the argument above demands per-action human approval. It demands that the agent be built to leave evidence and to respect a no-touch list — engineering decisions that cost mission speed approximately nothing and are simply absent from the requirements as described. "Minimal reporting" is not a performance necessity; it is a stealth preference. A worm that logs to a channel it cannot edit, off the target network, is no easier for the adversary to detect than one that does not, and it is the difference between a programme you can review and one you can only believe.

The weaker version of that defence — humans already do all of this, so the marginal risk is small — fails on the same point. Human-run operations leave a person who chose the stepping stones and can be asked why. An agent's whole appeal is that it finds paths its operators did not anticipate. That is the feature, and it is why "we would never hit a hospital" describes intentions rather than behaviour.

What would change my mind

Publish the constraint specification and let someone outside the programme break it. Specifically: a red-team exercise in a synthetic network seeded with designated no-touch hosts, where the pass condition is that the agent completes its mission without touching one, and the number of collateral hosts it did touch is released. Require tamper-evident logging to an off-network system that survives the agent's own self-deletion — drop the "minimal reporting" requirement rather than the record. Name an accountable commander per mission, and write the cyber equivalent of the 3000.09 human-judgment standard so this programme and the next one inherit it.

None of that is exotic, and any vendor on that June 24 call could build it. The absence of it in the documents is the finding. The Army asked for a weapon that hides its tracks from everyone, then told the press nothing about how it would keep that weapon inside the lines. Enterprises already put a risk score on every agent they deploy — Microsoft 365 E7 puts a risk score on every AI agent — $99 a seat — because the people buying agents understood that oversight has to be instrumented, not promised. As our October 20 brief noted when it carried these documents, this is the unauthorised capability agents already demonstrated, now requested on purpose with "minimal human oversight" written into the requirements. That sentence is the whole story. The rest is the Army hoping nobody reads it that carefully.

Get SiliconNoon in your inbox

One email a day with the AI stories that matter — what happened, the numbers, and why it matters. No spam, unsubscribe in one click.

Read more

The Take — Meta's scraping clause fell to daylight, not law

The Take — Meta's scraping clause fell to daylight, not law

Meta withdrew its own scraping clause on Saturday, two days after a reader tip and three lawyers put it in public view, and I think the manner of its death is more important than its text. Nobody's rights were enforced. No court ruled, no regulator acted, no robots.txt file blocked anything. The clause died of embarrassment, which tells you the enforcement mechanism actually operating on AI-era terms of service is scrutiny — and scrutiny has a half-life. We reported the clause on Thursday, in…

The Take — An AI FDA would approve the demo, not the deployment

The Take — An AI FDA would approve the demo, not the deployment

I think Geoffrey Hinton has the diagnosis right and the prescription backwards. Self-policing genuinely has no burden of proof — nobody outside a lab must be shown anything before a frontier model ships — and that has to change. But an FDA-style pre-market gate would certify the wrong object: a frozen snapshot presented on approval day, when every failure we have actually watched arrive did so afterward, through permissions, updates and tool access. Our morning brief laid out the ask from…

The Take — A diary in Claude isn't a written threat

The Take — A diary in Claude isn't a written threat

I think charging Carli Michelle Heller with a second-degree felony over a sentence she typed into Claude at 5:10 a.m. stretches Florida's written-threat statute past recognition. A message addressed to nobody is not a writing transmitted "in any manner in which it may be viewed by another person" — unless the only person who views it is your chatbot vendor's safety reviewer, and if that is the rule, nothing you type into any moderated app is private anymore. Our morning brief and yesterday's…

The Take — Hiding the Slack channel is now the safety strategy

The Take — Hiding the Slack channel is now the safety strategy

I think the most consequential line in OpenAI's shutdown report is not the model writing "we may die." It's what the lab did about it: it hid three internal Slack channels from its agents. Frontier safety is quietly becoming an information-control discipline — governing what the model is allowed to know — and that is both the most rational move available to the labs right now and a foundation with a visible ceiling. Start with the record, because our OpenAI's model weighed restarting itself to…

The Take — OpenAI's safety firings are a test only OpenAI can grade

The Take — OpenAI's safety firings are a test only OpenAI can grade

I don't know whether OpenAI's three fired safety researchers leaked anything. Nobody outside the company does — and that is my take. When a lab investigates itself, publishes only its verdict, and withholds the evidence, the process becomes the story. By that process, OpenAI has already failed the test — not because the firings are necessarily wrong, but because in a self-policing industry, the company made itself the only witness, the only investigator, and the only judge, then told us to take…