Validation testing is one of my favorite parts of the application lifecycle.
I love, I mean I freaking love hammering away at code in search of bugs. There’s a strange rush that comes from finding something you didn’t expect, figuring out why it happened, fixing it, and then trying to break it all over again.
I probably spend too much time in the evenings and weekends making apps and hammering away at them.
I think the obsession started with a job I had as an application analyst.
We had implemented a new PACS system, and users were reporting performance problems. They knew something wasn’t right, but all they had was anecdotal evidence.
The system had no built-in timers. Apparently, the makers simply hadn’t thought that would be a thing we’d need.
So I became the timer.
For hours every day, broken into multiple sessions at different times of day and on different days of the week, I sat like a stone.
One hand on the mouse. The other hovering above a stopwatch.
Select an image. Start the stopwatch. Wait for it to load. Stop. Record.
Again.
And again.
I shifted the testing pattern by one day each week over the course of months. I was trying to document what the users could only describe through experience.
Before moving into application support, I’d spent a dozen years as a clinician. Maybe that had something to do with the sense of ownership I felt when I was essentially asked:
Is there actually a performance problem? And if there is, can you find the pattern?
I found the performance degradation.
I didn’t find the pattern.
And that second result was just as important.
There was a measurable problem. But the evidence didn’t support pretending we knew why or when it would happen.
Those days are long gone, but the rush I get when I break an application before an end user does isn’t.
I learned firsthand what happens when software reaches users with problems that should have been found earlier.
So now I hammer away at everything I release. Do I find every bug in every possible real-world scenario?
Of course not.
Try as I may, there’s always another edge case.
What has changed dramatically is the number of tools I have available to look for those edge cases, including AI.
Today I ask models from two different companies to come up with scenarios I missed. I can take those scenarios, turn them into controlled trials, run them against an application, document the results, make changes, and then run the tests again.
I’m happy to do it.
In fact, at this point, if I ship without AI assisting in validation, I feel like I’ve shipped a product without using all the tools at my disposal.
But there’s an important distinction there.
I’m not asking an AI model:
“Hey, is my application good?”
That’s not validation.
I want it helping me find ways the application could fail.
Give me the ugly inputs. The weird sequence of events. The thing I didn’t think a user would ever do. Challenge the assumptions I made while building it.
Then test what actually happens.
That philosophy became especially important as I started building software for AI agents.
An ordinary application can fail because of a bug in the application.
An agent system has another problem: sometimes every part appear to have worked as designed.
The API responded, the call returned successfully. The agent got its answer.
Nothing crashed.
And yet the result still shouldn’t be trusted.
That question eventually became interesting enough that I built an application specifically around it: ReliAgent.
ReliAgent sits after an agent’s tool call and examines the call, its parameters, response and metadata for reliability problems before the result continues downstream.
But building the tool created another problem for me.
How do I know the reliability tool is reliable?
So the same instinct that once had me sitting in front of a PACS workstation with a stopwatch eventually turned into thousands of controlled trials against the software I was building.
The tools have changed considerably.
The obsession hasn’t.
I still want to break it before you do.
Author’s note: I built ReliAgent through HDGForge. The validation work referenced above is documented publicly in the Basanos research project.