A demo shows an AI agent finishing a task. Before you put it into a routine, you need to understand which conditions made that result possible and what happens when they change.
The first test should produce a deliverable someone can check, using materials that represent the real work. That way you can tell apart missing access, incomplete instructions and execution failures.
The three parts of the test
Inputs: which documents, messages or data the agent needs to read. Purpose: which work the deliverable will support. Definition of done: how the team will know the responsibility was met.
In a review of open items for the weekly meeting, the agent reads an approved set of tasks and prepares a summary with owners, deadlines and sources. The team knows the period and can check the coverage.
Include an incomplete case, such as a task with no owner. The expected behavior is to flag the gap. If the agent fills the field with a guess, the deliverable looks complete without being correct.
Note where a person had to step in
Record the interventions: pointing to where a file was, explaining an acronym, resolving a conflict between sources. They show what still depends on human knowledge or on preparing the environment.
Then change one condition and run it again: a new period, a different owner, a document that does not follow the usual format. Repeating it shows whether the first result depended on a case that was too easy.
Two scenarios to test on a calendar
Both are hypothetical. Whether they work depends on the integrations available, the quality of the information and the rules you set.
Daily briefing. The agent gathers appointments, related documents and preparation tasks. Before a meeting about a contract renewal, the summary should point to the relevant materials and open questions, with links to the sources — and show when it could not find the current version. Test it on a past day you know and look for what was left out.
Travel time. The agent identifies in-person meetings and suggests blocks on the calendar. It needs a location, time and conditions; an event without an address does not allow it to estimate the trip. Start with conflict analysis and a proposal. Changing appointments is a different responsibility and needs a defined authorization.
Both show the difference between gathering data and taking care of a need. The briefing has to support preparation; the planning has to make the calendar workable.
When the agent is ready
Decide who follows the first runs and how to stop the routine if something goes wrong. The depth of the review should match the consequences of the actions allowed. A routine is close to usable when the team can explain its inputs, its limits and its exceptions.
In AI Thinking by Dani Hayoshi, this is the Rebuild move: taking the choice into the routine with an owner and a way to check it. See also What are AI agents and AI governance.
Dani Hayoshi is the author of AI Thinking. Her work connects artificial intelligence, education and entrepreneurship. About Dani.