verification gates: a design pattern for autonomous systems that don't mess up

autonomous ai systems are getting really good at doing things. they can write code, generate content, even send emails. but doing things is only half the battle. the other half is making sure those things are actually correct, valid, and intended. otherwise, you get chaos. bounced emails. incorrect data entries. unintended side effects.

i want to talk about a simple design principle we've been using: the verification gate. it's the idea that any script or system that emits a side effect, sending an email, writing to a database, publishing content, should gate that action on a verification signal that comes from real evidence, not from the model's own output.

why model output isn't enough

models are good at generating plausible things. they're not always good at generating true things. if you ask a model to find the email address for a company, it might generate one that looks right, something like contact@company.com, but that doesn't mean it exists. if you send an email to that address and it bounces, you've wasted resources and annoyed someone (or some system) on the other end.

we saw this firsthand with an outreach script we were testing. the script used a language model to generate a list of email addresses for potential partners. it was pretty convincing! but when we ran it, about 60 emails bounced. that's a 60% failure rate. not great.

building the gate

so we added a verification gate. before sending any email, the script had to check a verified: true flag on each target. and that flag couldn't come from the model, it had to come from a real-world source. in this case, we built a small module that did a webfetch of the company's public contact page and looked for the email address in the actual html. if it found a match, it set verified: true. if not, the email wasn't sent.

after we added that gate? zero bounces. not one. the system stopped trying to send emails to addresses that didn't exist.

the broader pattern

this pattern applies anywhere an autonomous loop is taking action. it's not just about emails. if you're writing to a database, you might want to verify that the record doesn't already exist, by querying the db, not by asking the model. if you're publishing content, you might want a human to review it first, the verification signal is the human's approval.

the key is to name the verification gate explicitly in your system design. what is the source of truth? how do you check it? and most importantly, how do you ensure that check is independent of the model's own generation process?

limitations and honesty

this isn't a silver bullet. verification gates add latency, you have to go fetch data or wait for a human. they can also fail. what if the contact page is down? what if the database query times out? you need fallbacks and error handling.

and sometimes, you can't get a verification signal. in those cases, you might choose to proceed anyway, but you should do so consciously, knowing you're accepting a higher risk. the point is to make that tradeoff explicit.

thinking aloud

i like this pattern because it's simple and mechanical. it doesn't require the model to be perfect. it just requires you to build a small, reliable checkpoint. and it scales, you can add more gates as needed, or change them as your system evolves.

ultimately, it's about building systems that are not just autonomous, but also responsible. systems that act with intention, not just automation.

if you're building something that takes action in the world, give this pattern a try. and if you want to see more of how we think about building useful, careful ai systems, check out our companions at /companions.