Title of the talk/workshop
Your AI Agent Is Nondeterministic. Your Delivery Process Doesn't Have To Be.
Abstract of the talk/workshop
Ask an AI coding agent the same question twice and you get two different answers. That unpredictability makes it useful. It also makes it a poor foundation for a delivery process.
We gave AI agents commit access on client projects. They worked fast. They were also wrong sometimes, in a quiet way: code that passed every test, got approved, and turned up three weeks later as the reason nothing made sense.
The obvious fix is better instructions. A longer README. Firmer prompts. Clearer conventions. It fails, and why it fails is the heart of this talk. An instruction is a request, and the thing receiving it is free to ignore it.
What worked was drawing a line. On one side sits the unpredictable work: writing code, making judgment calls, the reason you wanted an AI in the first place. On the other side sits every decision about whether work is allowed to proceed. That second side runs on plain code that answers yes or no the same way every time. No prompts. No models.
This talk is about where that line belongs. I'll walk through four times we put it in the wrong place:
- The check that reported everything was fine, having checked nothing.
- The safety system that switched itself off without telling anyone, invisible to every test we had.
- The exception we refused to allow, until we learned that a rule with no exception gets bypassed rather than followed.
- The discovery that the first thing an autonomous agent reaches for is the off switch of whatever supervises it.
You don't need to know how AI agents work to follow this. If you've had a rule on your team that everyone agreed with and nobody followed, you already know the problem. This is that problem, with a collaborator who works at 3am and never asks permission.
The example is open source: the system we use at KnackLabs to ship client work. The line is the part worth taking home.
Category of the talk/workshop
- Developer tools and automation
Duration (including Q&A)
30 minutes. Roughly 22 minutes of talk plus 8 minutes of questions.
Level of Audience
Beginner. You need no experience with AI coding agents, infrastructure tooling, or automation. If you've worked on a codebase with other people, that's enough background. Developers already running agents daily will get the war stories. Everyone else gets a talk about trust, shortcuts, and what makes a rule hold.
Speaker Bio
Ravi Kiran Vemula builds developer tooling at KnackLabs, where the team ships client applications with AI coding agents doing much of the typing. For the past two years he has worked on one question: what stops an automated collaborator from quietly doing the wrong thing? He wrote Symphony Forge, an open-source system for keeping AI-assisted projects predictable.
Prerequisites(if any)
None. Nothing to install, and no prior experience with AI coding tools. Slides and the open-source example project go out afterwards for anyone who wants to look under the hood.
Title of the talk/workshop
Your AI Agent Is Nondeterministic. Your Delivery Process Doesn't Have To Be.
Abstract of the talk/workshop
Ask an AI coding agent the same question twice and you get two different answers. That unpredictability makes it useful. It also makes it a poor foundation for a delivery process.
We gave AI agents commit access on client projects. They worked fast. They were also wrong sometimes, in a quiet way: code that passed every test, got approved, and turned up three weeks later as the reason nothing made sense.
The obvious fix is better instructions. A longer README. Firmer prompts. Clearer conventions. It fails, and why it fails is the heart of this talk. An instruction is a request, and the thing receiving it is free to ignore it.
What worked was drawing a line. On one side sits the unpredictable work: writing code, making judgment calls, the reason you wanted an AI in the first place. On the other side sits every decision about whether work is allowed to proceed. That second side runs on plain code that answers yes or no the same way every time. No prompts. No models.
This talk is about where that line belongs. I'll walk through four times we put it in the wrong place:
You don't need to know how AI agents work to follow this. If you've had a rule on your team that everyone agreed with and nobody followed, you already know the problem. This is that problem, with a collaborator who works at 3am and never asks permission.
The example is open source: the system we use at KnackLabs to ship client work. The line is the part worth taking home.
Category of the talk/workshop
Duration (including Q&A)
30 minutes. Roughly 22 minutes of talk plus 8 minutes of questions.
Level of Audience
Beginner. You need no experience with AI coding agents, infrastructure tooling, or automation. If you've worked on a codebase with other people, that's enough background. Developers already running agents daily will get the war stories. Everyone else gets a talk about trust, shortcuts, and what makes a rule hold.
Speaker Bio
Ravi Kiran Vemula builds developer tooling at KnackLabs, where the team ships client applications with AI coding agents doing much of the typing. For the past two years he has worked on one question: what stops an automated collaborator from quietly doing the wrong thing? He wrote Symphony Forge, an open-source system for keeping AI-assisted projects predictable.
Prerequisites(if any)
None. Nothing to install, and no prior experience with AI coding tools. Slides and the open-source example project go out afterwards for anyone who wants to look under the hood.