I wrote last month about verification debt. AI made producing information cheap, but understanding it and checking it still costs real time and attention. Every agent run that nobody verifies adds to the debt. That debt is now turning into jobs, and the job has a name: verifier.
I have felt this shift in my own work. I build agents, and the hard part was never getting an agent to produce something. It was deciding whether what it produced was right, whether it was safe to keep, and whether the lesson was worth passing to the next run. When I skip that check, the mistake comes back, usually bigger. That is what verification is: the work of deciding what counts, before the next run builds on it.
The pattern is easy to miss if you only watch model releases. The models keep getting better at producing. The scarce skill is deciding what is true, what is good enough, and what should be kept. That is verification, and it is becoming a discipline of its own.
This is where the knowledge factory comes in. Harrison Chase put it plainly: the missing piece for agents is not a better model, it is better knowledge. A knowledge factory collects, curates, and maintains what a team knows, so every attempt makes the next agent smarter. But there is a catch I keep coming back to. Knowledge that was never checked is not an asset. It is a liability with a nice label on it. A factory that stores unverified attempts just automates the spread of mistakes.
So the factory needs a gate, and the gate is the verifier. The loop looks like this: a team tries something, the attempt gets recorded, a verifier checks it, and only then does it become knowledge the next run can reuse. The check is the step that makes the rest of the loop safe. Without it, you have a pile of attempts. With it, you have a factory.

The knowledge factory loop. An attempt is recorded, checked by the verifier, and only then becomes knowledge the next run can reuse.
What does a verifier actually do all day? Here is the shape of the work as I see it. You define the bar: take a real task, break it down, and write the standard a good result has to meet. You build the check: programmatic verifiers, tests, and scoring that run against agent outputs and catch shortcuts. And you investigate the failures: when a result looks right but is wrong, or a score can be gamed, you trace it back and fix the check. It is part engineering, part judgment, and the judgment is the part that cannot be automated away.
There are two kinds of checking in that gate, and a serious team needs both. Automated verifiers run the checks at scale, the same way a test suite guards a codebase. Human verifiers are the domain experts who set the bar the automated checks measure against. The human decides what good looks like. The machine checks it at scale. One without the other either drifts or stalls.
That is what we are building with Attempted AI. We started with the record: what your team tried, what happened, why, and what changed. The record is the raw material. The verifier is what turns it into knowledge worth keeping. Agent failures and security attempts go first, because those are the attempts where an unchecked lesson costs the most.
If you are deciding where to focus next, here is my read. Do not only learn to build agents. Learn to verify them. Learn to write the check that says this output is right, this run is safe, this lesson is worth keeping. The builders will keep multiplying. The people who can say, with evidence, that the work is good, those are the ones every team will need.
The factory runs on knowledge. The verifier decides what counts.
Dig deeper
- The Hidden Cost of AI: Verification and Comprehension Debt (my earlier article on verification debt)
- Attempted AI (where we are building the record)