The problem I actually hit
I built an automation pipeline that scrapes job listings, matches them
against a resume, and auto fills applications on supported ATS platforms
like Ashby, Workable, Lever and Greenhouse.
It worked fine until it hit a listing on a platform it didn't recognize.
It just skipped, correctly, because it had no idea how to fill an
unfamiliar form. Rule based automation is precise, but it only knows
exactly what you taught it. Nothing more.
The obvious fix is to add an LLM agent that can read any form and fill
it in. But that brings its own problem. Agents hallucinate, misclick,
or submit wrong data with full confidence. Replacing deterministic
logic with a fully autonomous agent just trades one failure mode for a
worse one. Instead of an honest skip, you get a silent, confident
mistake.
The idea: converge, don't replace
So instead of choosing rules or agents, I'm restructuring the pipeline
so each one does what it's actually good at. The rule based path stays
the default. It's fast, deterministic, and has basically zero
hallucination risk, and it handles every case it was explicitly built
for.
The agentic path only kicks in as a fallback, when the rule based path
fails to recognize the pattern. It's scoped narrowly too. The agent
isn't told to fill out the form, it's asked to identify the purpose of
a specific input field and return a structured answer, not take a
free form action.
That turns the agent from an autonomous decision maker into a
constrained classifier with one narrow job, which behaves very
differently in terms of reliability.
Why this actually reduces hallucination
A general purpose agent tends to hallucinate more as its task gets
broader, since there's more room for it to improvise instead of being
correct. By keeping the rule based layer as default and only calling
the agent for one narrow sub task, like figuring out what a single
input field represents, you shrink the space where the model can get
it wrong.
Two things help keep this safe in practice. The agent only returns
structured, typed output, never free text it can improvise around. And
below a certain confidence threshold, it doesn't guess at all, it just
flags the field for manual review, same as the rule based skip does
today.
What I'm not claiming
This doesn't make the agent hallucination proof. It just reduces the
blast radius of a wrong guess and keeps the system's default behavior
deterministic. I'd rather have a system that's mostly rules with a
narrow, well fenced agentic layer, than one that's fully agentic and
occasionally wrong in ways I can't predict.
Next step
I'm prototyping this with LangGraph as the state machine. Rule based
node runs first, and the agentic node only triggers as a conditional
edge on failure. Playwright still handles the actual browser
interaction either way. I'll follow up with real numbers once I
actually have them, not just the theory.