DuetUI: A Bidirectional Context Loop for Human-Agent Co-Generation of Task-Oriented Interfaces

Yuan Xu, Shaowen Xiang, Yizhi Song, Ruoting Sun, Xin Tong

Abstract

DuetUI introduces a bidirectional context loop where an AI agent scaffolds interfaces by decomposing user goals, while direct user manipulations continuously steer the agent’s next steps. Through formative studies and a 24-participant evaluation, the system delivers higher task efficiency and alignment compared to single-shot generation baselines.
arXiv Demo Video
DuetUI bidirectional context loop

Key Contributions

  • Defines the human-agent co-generation paradigm grounded in a bidirectional context loop.
  • Implements DuetUI, enabling agents to decompose tasks into editable scaffolds that welcome user overrides.
  • Demonstrates significant gains in task efficiency and interface usability through a controlled user study.

How the Bidirectional Loop Works

1. Goal Decomposition

The agent interprets high-level intents and produces an interface scaffold with structured layout, task modules, and actionable suggestions.

2. Direct Manipulation Feedback

Users adjust components directly on the canvas. These manipulations provide implicit feedback that the agent converts into updated context.

3. Iterative Co-Generation

With refreshed context, the agent proposes refined UI states or complementary components, sustaining a fluid collaboration rhythm that respects user agency.

Contact

No.1 Du Xue Rd, Nansha District, Guangzhou

Sources

Follow Us

  • Bilibili
  • Wechat QR code

Copyright © . All Rights Reserved. — ARK Lab • License