Making the AI iteration loop a connected experience.
AI teams were building in notebooks, evaluating in scripts, and comparing results across scattered tools. I worked on connecting these steps in one visual workflow.
- Focus
- AI workflow tooling
- Design work
- Authoring, evaluation & comparison
- Users
- Engineers, data scientists & PMs
Make the workflow itself visible, so every edit, test, and result stays connected to the same logic.
The work was iterative. The tools were disconnected.
Each experiment produced more scripts, configurations, and results to keep track of. As flows grew, teams struggled to trace what changed and understand which version worked best.
Logic was hard to follow
Engineers had to piece together execution paths across code and configuration panels.
Evaluation did not scale
Data scientists relied on one-off tests and manual scoring instead of a repeatable comparison workflow.
Decisions were hard to share
Product managers lacked a shared view of prompt performance and the reasoning behind changes.
Start with how teams actually experiment.
The research combined product logs, expert interviews, and contextual inquiry. It pointed to a recurring issue: flexibility without a shared structure made the work difficult to reproduce.
Authoring happened in local notebooks. Evaluation relied on ad hoc scripts. Deployment lived in a separate pipeline. Improving any one screen would leave those handoffs intact.
How could teams experiment freely while keeping the logic, evaluation, and results understandable to everyone involved?
One workflow model. Three connected activities.
The foundation was a directed acyclic graph: a visual model of the steps in a flow and how data moves between them. Authoring, testing, and comparison could then use the same structure.
Keep execution logic visible while authoring.
A graph view shows how nodes connect; inline configuration and node-level inspection let users follow the flow while making changes. Engineers can inspect a single step without reconstructing the entire execution path.
Bring testing and comparison into the authoring loop.
Bulk testing and evaluation share a configuration flow. Results become comparable objects, making it easier to review variants against the same metrics and decide what to try next.
Preserve context through collaboration and deployment.
A shared flow gallery and connected deployment path carry the same workflow forward. Teams can reuse a flow and understand its structure without piecing together a separate handoff.
Reduce the work of understanding the interface.
A clearer hierarchy for larger flows.
As flows grew, readability became the bottleneck. The iteration separates inputs and outputs, strengthens the node hierarchy, and gives the graph a clear place in the workspace.
One setup flow for testing and evaluation.
Previously, Bulk Run and Evaluation required separate setup. Bringing them together removed repeated configuration and reduced context switching.
Results that can be compared directly.
Evaluation results were scattered across runs. A common result structure supports side-by-side review of different variants.
A more connected way to experiment.
Less tool switching
Authoring, bulk testing, and comparison sit within the same workflow.
Less repeated setup
Shared configuration carries context between testing and evaluation.
A reusable foundation
The workflow pattern was incorporated into Azure AI tooling.
The biggest design contribution was connecting the loop. A shared workflow model made individual features easier to understand, and gave teams a more consistent way to build on each experiment.
What if editing stayed on the canvas?
This exploration is separate from the product work above and was not a shipped Microsoft product change.
The next question I explored was how to keep editing anchored to the graph. Selecting a node opens an inspector in context, so users can adjust it while keeping the surrounding flow in view.
