Human and AI interaction case study
AI in the Human Loop:
Designing & Wireframing sustainably and efficiently An exploratory HCI study of speed, traceability and human authority
As a UX researcher investigating how AI can reduce repetitive production work without taking design authority away from the person accountable for the outcome, I designed a pipeline that turns app maps, screenshots and my evaluation notes into editable text wireframes. Every proposed change is traced to a note or rule, fixed checks report changes and the logic behind them, and the workflow stops at decisions that require human judgment.
6 projects
generated for data collection and rule validation
8 test runs
using the same handover, model, tools and rules with different execution plans
36+ rules
general rules for every project, plus 8 for the banking domain
The HCI question
How can AI accelerate wireframe production while preserving a designer’s agency, accountability and ability to understand every change?
This project mentions generation of screens from 3 different projects: VBSP, PB Engage, Vietnam Airlines.
The original problem was the time consumption of creating wireframes to communicate across teams. A project could need a research review and wireframe generation in 1 week, especially during the age of AI where everyone expects work to be fast tracked. I started off with connecting Wiretext MCP to Claude Code, generating every wireframe manually from my notes and the original png and prompting screen by screen. As the need for this tool increased and the time distributed for design decreased, I saw a need to make a pipeline that held my decision making and logic in a format that UI Designers can read.
What I investigated
Where human approval belongs, how decisions can remain traceable, and how different execution plans change time and computational cost.
What I gave the pipeline
Rules spanning position of components, logic for what components to use based on the nature of the app, the app map, and notes.
Original screen
Wireframe from the tool
New design in Figma
How the workflow distributes authority
I give the tool three things: a map of the app, a screenshot of each screen, and my evaluation notes. From there, the pipeline takes over. The AI describes and proposes. I interpret the findings and decide whether the work continues. I retrieve the md file of screen content and Wiretext link to edit by hand. After that, the product is uploaded onto a GitHub repository of choice, but before that it is stored locally on my computer for quality control before any changes are pushed.
- MeHandover. I give the tool the app map, screenshots and my evaluation notes.
- ToolFirst pass. The tool checks whether required inputs are present and asks when something is missing.
- AIDescribes the screen. The AI records what the screenshot appears to show, by element.
- My rulesCheck the original. My general rules and project specific banking rules run over that description and flag possible problems. No generative model makes a decision in this step.
- AIRedesigns and explains. The AI proposes a redesign in my format and traces each change to a note or rule.
- ToolAutomatic checks. Eight programs look for missing or added elements, unanswered notes and format errors. They report discrepancies rather than certifying usability.
- MeMy decision. I approve, ask for changes, or skip. A correction I make becomes a project rule for later screens inside the current project, or a general rule if I state so.
Two modes, three levels of rules
I built two modes. Careful, the default, draws one screen and waits for my decision before the next. Fast draws one sample screen, waits for my approval, then draws the rest at once. Careful spends fewer tokens and gives me a viewer for each screen. Fast finishes in minutes and gives me one long review.
Both modes work from the same three levels of rules. General rules apply to every project. Domain rules apply to one kind of product (banking, airlines…). Project rules are the ones I state while working, and each applies to every screen drawn after it. That is how the tool takes my fixes into account without a new round of instructions: a project rule is written into the project’s context file, each later helper reads that one section, and a correction costs a few lines rather than a longer conversation. It is also how a run recovers from a mistake.
Designing the boundary of control
The AI does the drawing, but the central design work was deciding how far to trust it. I separated generation, verification and judgment so that fluency could not be mistaken for correctness. The general checks are deterministic and cannot be silently bypassed; project-specific exceptions must be stated and recorded.
The AI may do alone
- Describe what each screenshot shows
- Redesign each screen and explain every change it made, against my rules and my notes
- Draw the wireframe in the format I designed
- Propose new rules, which wait for my yes
The system must check and report
- Possible elements that were dropped or added
- Whether every evaluation note received a response
- Whether the drawing follows the approved format and recorded rules
- Whether every required check ran; missing checks stop the workflow
It must stop and ask me
- Something in my handover is missing
- It is unsure what the screen shows or means
- A drawing would go to Wiretext or any outside service
- Any approval. I have the last say on its output.
Five design decisions
These decisions shaped both the tool and the research questions it raised.
Decision 1
Constrain autonomy before optimizing speed.
The tool may work through repetitive production tasks, but it has limits on run time, computational use and what counts as approval. It exists for time-bounded design sprints, not as a replacement for a designer.
Decision 2
Choose an output that remains inspectable and editable.
I chose Wiretext because plain text can be reviewed, versioned and changed without a proprietary visual editor. The Markdown file also preserves the screen’s structure, rationale and interaction logic for later work. More on that under Open Design.
Decision 3
Make local conventions explicit instead of assuming a universal default.
The Vietnamese banking and travel interfaces in my test projects used language, formatting and interaction conventions that general-purpose generators often failed to preserve. I encoded the conventions I observed as domain rules.
Decision 4
Keep the designer accountable for interpretation.
AI proposes; the designer remains responsible for deciding whether a change answers the research evidence and fits the context. Editable text, explicit rationale and recorded approval points make that responsibility visible rather than hiding it behind a polished generated screen.
Decision 5
Collect evidence for the next question.
The output is documentation as well as a wireframe: inputs, proposed changes, rule triggers, approvals, time and computational cost. That record makes the workflow open to critique and gives a future multi-designer study something concrete to test.
What it makes
A wireframe in text form (example shown is from the Vietnam Airlines booking app), one of my test projects, drawn from one screenshot and four of my sticky notes. Everything in gold is a change the tool made, and for each one it wrote down which note or rule asked for it. The format is mine: the marks, the phone sized frame and interactive format visualization (buttons, tabs,…) all come from the general and project specific rules I stated.
Exploratory evaluation
I compared three executions of the same handover: the same app map, original screenshots and notes for 15 money transfer screens in PB Engage Vietnam. One run was careful, one fast, and one hybrid, two careful screens followed by fast. Because I was the only reviewer and the runs were sequential, I treat this as a process comparison, not a controlled usability experiment.
The comparison exposed a useful trade off. Careful used the fewest helper tokens per screen but took the longest from start to approval. Of its 262 minutes, 46 were the machine working and the rest were mine. Fast did its machine work in 12 minutes and then waited about an hour on me. Hybrid finished soonest, 48 minutes with 26 of them machine work, and used the most tokens without an observable benefit in this run. A project rule stated once on an OTP screen, to omit the phone’s own keyboard from the drawing, was applied consistently to later screens in all three runs.
Three runs against the clock
- Careful
- Fast
- Hybrid, two careful screens then fast
- the hollow dot is my last approval
Each dot is one helper reporting back, placed by the minute it reported and the tokens spent by then. The dashed tail runs on to my last approval. Fast had its drawings in 13 minutes and waited an hour for me. Hybrid finished soonest at 48 minutes and spent the most. Careful spent the least and ran to 262 minutes, because it stops for me after every screen. Every time here is time on the clock.
Context a screen, as the helpers read it
What each helper reported when it stopped is the size of what it was holding, not a bill. Careful’s helpers held the least a screen, and with my own estimate of the main chat added, fast is lowest. Reading the session records afterwards, the fast run’s helpers processed about 52 million tokens between them, because every turn re-sends the conversation.
Helper runs it took
Runs with more helpers used more tokens, though I did not measure what they spent them on.
The three runs, side by side
Same project, same day, same tool, same reviewer. The plan is what I changed on purpose. The order of the runs and the AI’s write-ups changed too. Every time on this page is time on the clock, from the handover to my last approval. The machine working is each helper timed from the work order that started it to the moment it reported back, with overlapping helpers counted once. The pipeline does not record that figure, so it is a reconstruction from the timestamps, good to a few minutes either way.
| Measured | Careful | Fast | Hybrid |
|---|---|---|---|
| Plan | careful | fast | two careful, then fast |
| Ran | second | first | third |
| Screens approved | 15 of 15 | 15 of 15 | 15 of 15 |
| Screens sent back | 1 | 0 | 1 |
| Helper runs | 17 | 28 | 35 |
| Helper context readings, added up | 1,682,568 | 2,571,396 | 2,893,613 |
| Context a screen | 112k | 171k | 193k |
| Whole run, my reviewing included | 262 min | 74 min | 48 min |
| The machine working | 46 min | 12 min | 26 min |
| Waiting on me | 216 min | 62 min | 22 min |
| Questions the AI asked | 3 | 2 | 6 |
| Problems the rules found | 57 | 105 | 75 |
| This chat, my own estimate | about 2.5M tokens | about 1.5M tokens | about 2.4M tokens |
What the evidence supports
For this project, careful minimized helper tokens and fast minimized elapsed time. Hybrid added cost without an observable process benefit. The traceability mechanism also worked as intended: one recorded project rule propagated to later screens.
Limits. One project, one reviewer and one run of each plan mean order effects and my growing familiarity are mixed into every difference. My approval is not evidence of user usability. The main coordinating chat was estimated rather than measured, and energy use was not measured.
Next study. I would ask multiple designers to use the workflow, examine where their rules agree or conflict, and evaluate selected outputs with representative users. That would test both designer agency and user outcomes instead of treating either as assumed.
What came after: Open Design
My first banking routine became the seed of Open Design, the tool my department’s AI team built. It reads a project’s documents and carries them to a text wireframe and a usability check before anyone opens Figma. The goal is to go on from there to screens built from the team’s own components. I shaped the two steps where judgement lives: the research criteria each screen has to meet, and the heuristic evaluation that scores the wireframe before UI is built.
- ToolDocs to Markdown. Pulls Confluence and JIRA sources into clean Markdown: every screen as a table of fields, with type, interaction, whether required, and length.
- ToolSystem map. Sorts the documents by app and records the handoffs between apps, one run for the whole project, editable by hand.
- AICustomer journey. Maps the end-to-end journey from the documents, with the source text behind each stage.
- My rules + AIUX research. Desk research against a UX knowledge base, NN/g, Baymard and Growth.Design, produces the criteria each screen must meet, tagged must, should or bonus, each with its reason, its journey stage and its source. This is where my rules went. There are also open source rules we allowed the LLM to take from.
- AI + my rulesUX spec. Writes the specification for each screen from the features and the journey, with a text wireframe in my format at desktop, tablet and mobile widths, and a link to open it in Wiretext.
- My rulesHeuristic evaluation. Generated screens and the logic behind them.
Every run leaves the same kind of record the wireframe pipeline does: the criteria, their sources, the findings and the cost. The UX spec run shown here took 8 minutes 10 seconds and about 40,000 output tokens. The product is an accounting system and the company data in the screens is sample data.
Reflections
Every number here is a cost or a count. None is a measure of quality.
That is the next study. Designers who have not seen the scores rate the same wireframes, and I learn whether the pipeline’s judgement predicts theirs. Until it does, I will not call a score a result.
The rules are the research.
What I want to study is how to formulate information so that both people and language models understand it, and how to do that with the least energy. The 36 rules on this page are a first draft of that question, and there are many more to find.
Autonomy and AI in the Human Loop
I built this project for an AI to sit in phases of a Human Loop, not controlling the entire operation. I’ve learned that understanding what LLM built is important to the design process, and to vibe code with guardrails in place.