Factory's Droid saved me from a huge security mistake

My coding agent wrote a security hole this week, and I almost committed it.

I was adding a new badge to learnmcp, my open source learning framework. The Twilio track needed one that checks you've set up your credentials properly: account SID and auth token (or API key and secret) present in your environment, not hard-coded in your source.

That check needs exactly one fact from a .env file. Which variables are set. It never needs the values. The server only ever asks "is TWILIO_ACCOUNT_SID set?" and has no business seeing the SID.

The agent's first pass didn't see it that way. It read your .env and was set up to hand the whole thing to my server, secrets included, so the server could work out which names were there. The tests passed. It was one git commit away from going in.

It didn't ship because of the pre-commit hook.

The gate

Every commit in this repo goes through a git hook that sends the staged diff to Factory's Droid and asks for a VERDICT: PASS or VERDICT: FAIL. A FAIL blocks the commit. I wrote up how that hook works separately, but the short version is that it strips sensitive paths from the diff, runs a secret scanner first, and lets the reviewer see only the diff, never the repo. If the reviewer itself crashes, the commit goes through. A broken tool shouldn't lock me out.

So Droid read the diff, failed the commit, and I had the agent fix it. The obvious fix was to pass only the variable names instead of the contents, parsing each line for NAME=.

That's where the real work started. The hook blocked the commit nine times in eighteen minutes.

#	What Droid found	Severity
1	A line inside a multi-line quoted value, like a private key, can look like NAME=. Padded base64 (YWJjZA==) got reported as a variable name.	High
2	Escaped apostrophes in single-quoted values closed the quote early and reopened the same hole.	Medium
3	Keys with dots or hyphens (TLS.KEY="...) skipped my quote tracking entirely.	High
4	YAML-style NAME: "... assignments did the same.	High
5	A value ending in two backslashes and a quote was read as closed.	High
6	The CLI path kept any content already attached to an .env signal.	Medium
7	Windows paths like config\.env.local weren't recognized as env files on a POSIX server, so they skipped the scrub completely.	High
8	A missing working directory skipped redaction altogether.	Medium
9	Editing a .env through the CLI dropped the content and the key names, so new keys went undetected.	P2

Number nine is a plain functional bug. The other eight are ways the "fixed" version would still have let secret text through.

I'd have missed several of those. A few I'd only have found once somebody's secret turned up on my server, and I do not want to learn about Windows paths that way.

The tenth attempt passed.

Droid reviews. It doesn't fix.

Droid can fix things, and I've deliberately not let it. When I set the hook up I made a rule: it finds problems, my coding agent fixes them, and I read every change before it lands. I wanted a stretch of watching what it flagged and whether I agreed before giving it any autonomy.

A run like this one changes my mind about the destination. Nine real findings in eighteen minutes, several of which I'd have missed. That's too much value to keep doing by hand forever. The right direction is automation, so the loop of review, fix and re-review runs without me sitting in the middle of it.

What I won't compromise on is who checks the work. The agent that builds a fix shouldn't be the one that decides the fix is good. Same harness, same blind spots, same confidence. That's how the "fixed" parser kept passing my tests while leaking. I already use a fix and verify pattern for this: one agent writes the change, a separate one with fresh eyes verifies it. Droid is a natural fit for the verifier side, because it never saw the code being written.

So the plan is to let automation do the fixing, but never let it grade its own homework.

What it cost

It cost me time. I figured this was a five-minute job. It took about twenty. Each block meant reading the finding, fixing it, rerunning tests, committing again, and waiting a minute or two for the review. Some of the later rounds felt like the reviewer kept moving the goalposts.

The five-minute version was a security hole sitting in a green test run, headed for everyone who installs the plugin. It could have passed pieces of private keys to my server while I believed the problem was solved. So the trade was fifteen extra minutes against a leak I'd have found when someone else did.

Fifteen minutes is cheap.

Summary

All I wanted was to know whether a variable was set. The first version of the badge would have handed my server the values too, and my tests would have called that a success.

Three things stuck with me:

• Green tests aren't a review. The agent's code did what I asked. It still leaked, and the leak lived in a place my tests never looked.
• The builder shouldn't verify its own work. Droid caught what the coding agent missed because it only saw the diff, with none of the agent's assumptions.
• Automate it, but keep the verifier independent. I'm moving toward letting agents fix what Droid finds. The part I'm keeping is that something other than the fixer decides when the fix is good enough.

If you build with AI coding tools, put an independent reviewer between the code and your main branch. Mine is a plain git hook (.githooks/pre-commit), opt-in through core.hooksPath. It cost me fifteen minutes and it's the reason an agent's mistake stayed a mistake.

Subscribe at quintonwall.com and follow me on YouTube at @seeqcode for more.

Written with Claude Code. The pre-commit review is Factory's Droid.