Rethinking the Interface: From Operational Surfaces to Agency Architecture

Interface and User Interface

When people talk about UI today, they usually think of colors, typography, icons, layouts, components, or the role of a UI designer. User Interface has gradually become synonymous with Graphical User Interface, which has in turn become synonymous with visual mockups in Figma.

But if we return to the original meaning of interface, an interface has never been merely a visible “surface.”

An interface is fundamentally a relationship.

It exists between two systems with different internal structures, allowing them to connect, exchange, and collaborate without either side having to understand the other’s entire complexity. It is both an organized boundary and a set of representations, mappings, and protocols that take effect on both sides of that boundary.

Code still preserves this older meaning. In TypeScript, for example, we might define:

interface PaymentService {
  preview(request: PaymentRequest): Promise<PaymentPreview>;
  execute(request: PaymentRequest): Promise<PaymentResult>;
  refund(paymentId: string): Promise<RefundResult>;
}

This interface is not a visible surface, and it does not specify where a button should appear. It defines what a component promises to provide, what a caller must pass in, and what result the caller can expect.

The caller does not need to know which bank the payment service connects to, how it stores transaction records, or how many internal steps a refund requires. The implementation does not need to know whether the caller is a website, a mobile app, an automation script, or an Agent. Both sides collaborate through the same contract while keeping their internal structures independent.

An interface contains at least four things:

Boundary: which capabilities belong inside the system and which are accessible from outside;
Abstraction: the understandable, actionable model through which the system presents itself;
Protocol: which inputs are valid, what they mean, and how the system will respond;
Feedback: whether an action was received, how state changed, and how to recover after failure.

An interface is therefore not a static object, but a continuously running loop:

Input → state change → output → judgment → next input

Buttons, forms, and pages are merely graphical expressions of this loop. What is actually being designed is how the two sides of a boundary understand and affect one another, and how they confirm that an effect has taken place.

User Interface is a special case of interface: it connects people and computing systems.

One side is human. People have goals but cannot always express them precisely. They rely on perception, experience, and memory; they hesitate, misunderstand, and make mistakes. They also need a sense of control and safety.

The other side is a computing system. It depends on explicit inputs, changes state according to rules, and contains data, permissions, exceptions, and many internal processes hidden from the user.

The core work of UI is to translate in both directions between these two very different worlds:

Translate human intent into operations the system can execute; translate system state into information a person can perceive and understand.

So the real design questions behind a button are not only its color, radius, or shadow:

Does the user understand what it means?
Can it be executed right now?
Which objects will it affect?
Has the system received the action?
How long will it take?
What happens if it fails?
Can the result be undone?
Who has permission to execute it?
Who is responsible for the consequences?

The screen is UI’s most common medium, not its essence. Command lines, voice, notification sounds, and vibration can all form User Interfaces. Even in the broader world of designed artifacts, a steering wheel and a door handle perform similar translation work. A page is only a snapshot of an interface at a particular moment; a complete UI exists in time and includes state changes, waiting, feedback, errors, recovery, permissions, and history.

UI is also the external expression of a product’s internal model. Projects, tasks, owners, states, priorities, and deadlines may look like fields on a page, but they are really the system’s definition of “work.” Data models, permission models, business rules, and state machines are ultimately perceived by users through the UI.

Many so-called experience problems are therefore not caused by poor visual design, but by a mismatch between the system model and the user’s mental model.

More importantly, interfaces are never neutral. They decide what is visible and what is hidden; what is easy and what is deliberately difficult; who can act and who can only observe; what happens by default; what can be undone; and who bears the cost when the system makes a mistake.

UI is a property of a product, not a deliverable owned by a design department.

A Brief History of User Interface

In early batch computing, people submitted jobs through punched cards, paper tape, job descriptions, and operators. An interface between people and computers certainly existed, but it consisted of input media, programming languages, operating procedures, and organizational workflows. Interaction was not real-time, and feedback might arrive hours later.

In the 1960s, time-sharing systems and interactive terminals began allowing people to communicate directly with a running computer. A person entered a command, the system returned text, and the person entered the next command based on that result. This became the Command-Line Interface (CLI):

User intention
→ symbolic command
→ system execution
→ textual response
→ next command

CLI turned the relationship between people and computers into a frequent, continuous feedback loop. It used language as its abstraction layer: people did not need to manipulate memory, circuits, or devices directly, only learn a set of commands, parameters, and composition rules.

This interface is highly expressive. Commands can be composed, repeated, saved, and automated. The same action can be entered by a person or invoked by a script, and the process naturally leaves a textual record.

But CLI also requires people to remember command names, syntax, object paths, and system state. What the system can do is usually not visible by default; it is hidden in manuals and in the user’s memory.

The basic relationship of CLI can be summarized as:

People describe actions in the language specified by the system.

At the same time, another route for human-computer interaction was emerging.

In 1963, Ivan Sutherland’s Sketchpad already allowed people to create and manipulate screen graphics with a light pen. In 1968, Douglas Engelbart’s team demonstrated the mouse, windows, hypertext, and collaborative documents in what later became known as the Mother of All Demos.

The Xerox Alto, designed in 1973, combined a bitmap display, mouse, windows, menus, and direct manipulation in a personal workstation. Xerox Star, Apple Lisa, and the 1984 Macintosh gradually moved graphical interaction from research environments into commercial products and the hands of a much wider audience.

Graphical User Interface (GUI) did not simply add icons to commands. It changed how the system presented itself:

CLI
enter symbols → invoke capability → read result
 
GUI
see objects → select and manipulate directly → observe change

In a CLI, a file is primarily referenced by a name and path. In a GUI, it can appear as an object that can be selected, dragged, and placed into a folder.

In a CLI, users must recall commands. In a GUI, available actions can be recognized through menus, buttons, and controls.

In a CLI, the system is more like a language that must be learned. In a GUI, the system is more like a visible world made of objects, spaces, and rules of operation.

The fundamental difference between CLI and GUI is therefore not merely text versus graphics. It is how interaction complexity is distributed between people and systems.

CLI provides a compact and powerful symbolic interface. The system does not need to display all of its capabilities or turn every state and operation into a visible control. In return, people take on more work:

remember which commands exist;
remember command names, parameters, and composition syntax;
maintain objects, paths, and current state in their heads;
break a goal into a sequence of correct operations;
infer the next step from textual output and errors.

The system provides a language, while the person must understand the world behind that language. Much of the interaction complexity lives in memory, experience, and mental models.

GUI takes on more translation work. It must externalize available objects, legal actions, current state, parameter ranges, and operation results:

files become icons;
hierarchy becomes folders and windows;
capabilities become menus, buttons, and controls;
parameter constraints become options, ranges, disabled states, and validation;
execution becomes selection, loading, progress, and animation;
errors and recovery paths become notices, undo, and rollback entry points.

CLI therefore relies more on recall, while GUI relies more on recognition. CLI asks people to adapt to the language specified by the system; GUI asks the system to translate itself into a world people can perceive and manipulate.

This does not mean that CLI hands all complexity to people, or that GUI exposes all of the system’s complexity. A command is already an abstraction that hides hardware, processes, and implementation details. A GUI also does not show the true structure of a system; it presents a selected, compressed, translated operating model. Good GUI design does not expose everything. It lets the system bear the cost of representation, constraint, and feedback so that people do not have to maintain all of that complexity alone.

Complexity does not disappear. Complexity that cannot be eliminated is redistributed among people, interfaces, and systems:

Some complexity is remembered by people, some expressed by the interface, some handled by the system, and some hidden until an exception or error occurs.

The development of GUI can be understood as a transfer of responsibility for complexity. System implementation and interface design become more complex in exchange for lower human learning, memory, and operation costs.

But GUI’s success also created a conceptual side effect:

Interface, a relationship, was reimagined as a surface on a screen.

As personal computers, desktop software, the Web, and mobile applications became the main ways most people encountered computing, GUI almost monopolized everyday ideas about UI. Pages and controls are easy to see, screenshot, review, and deliver. Capability boundaries, state machines, permissions, errors, and feedback protocols retreat behind the visual surface.

Over time, User Interface became equivalent to GUI, and GUI became equivalent to visual presentation. A concept describing the relationship between people and systems gradually became a visible artifact and a job category.

In reality, CLI and GUI are not two successive stages in which one replaces the other. They are two interfaces that continue to coexist:

CLI organizes system capabilities into a callable, composable language;
GUI organizes system state into perceptible, directly manipulable objects.

GUI is better for discovery, comparison, spatial organization, and continuous feedback. CLI is better for precise expression, repetition, batch processing, and automation. Development, operations, and infrastructure have never truly left CLI behind.

This history reveals not that graphics replaced commands, but how systems use different abstractions to turn their capabilities into forms people can understand and use—and who bears the complexity of interaction. Batch computing depended on programs, media, and organizational processes to arrange work in advance. CLI gave people real-time control while requiring them to remember a language, maintain state, and plan steps. GUI made the system take on more representation, constraint, and feedback work to reduce human operating costs. In the process, UI’s broader meaning was obscured by its most successful medium.

Every interface answers the same question:

To make action possible, which complexity should people carry, which should the system carry, and which should the interface absorb?

The arrival of Agents will change this distribution again.

The Emerging Interaction Triangle

In traditional software products, interaction is usually modeled as a binary loop between a user and a system:

Human ↔ System

The user expresses intent; the system accepts input, changes state, and provides feedback. Even when the system contains extensive automation, it is usually wrapped inside one unified “system” from the user’s perspective. The user remains the primary decision-maker and operator.

Automation and software agents are not new. The important change is that an Agent is beginning to appear as an explicit actor that can be delegated work, form plans, and act on real systems. The binary model is no longer enough to describe the full relationship.

An Agent is not merely a new control in the UI or a smarter search box. It is an actor granted limited powers of perception, judgment, and action. It can understand a goal, form a plan, select tools, execute multiple steps, and change the state of a real system along the way.

Some work once performed by the user—discovering capabilities, remembering an operating language, decomposing goals, organizing steps, and deciding what to do next from feedback—can now be delegated to an Agent. People no longer need to specify every action themselves. They begin to specify goals, boundaries, and judgment conditions instead.

Complexity has not disappeared. It has moved from human operation into the Agent’s understanding and execution, into how the system expresses its capabilities to the Agent, and into how people authorize, supervise, and verify the process.

We therefore need to separate the agency previously hidden inside the “system” and model product collaboration as an interaction triangle:

The triangle contains three edges that all need design:

Human ↔ Agent: how people express goals, constraints, and authorization, and how the Agent interprets, reports, and requests judgment;
Agent ↔ System: how the Agent observes the environment, discovers capabilities, calls tools, changes state, and handles errors;
Human ↔ System: how people independently inspect real state, verify results, operate directly, take over, and roll back.

This is not a simple User → Agent → System chain.

If the user can reach the system only through the Agent, the Agent becomes the sole operator, the sole interpreter, and the sole source of the claim that “the work was completed correctly.” The user no longer sees system facts, only the Agent’s account of those facts.

The triangle preserves the Human ↔ System channel of fact and control, independent of the Agent’s narration. “Direct” here does not mean without an interface or intermediary. It means that people can read authoritative system state, verify results independently, and intervene when necessary. The Agent cannot control or falsify this channel, so it becomes a check and a safety edge in the Agent system.

Three Edges, Three Design Tasks

Human ↔ Agent: The Delegation and Governance Interface

Traditional UI usually asks the user to specify how something should be done:

choose region → choose size → enter name → click deploy

Agent interaction lets the user express what they want:

Deploy a test environment in Shenzhen,
keep the monthly budget below ¥500,
do not expose it to the public network,
and reuse existing resources where possible.

The user no longer decides every operation. They delegate a goal and part of the agency to an Agent. The design problem is not primarily a chat box, but a delegation contract that forms over time:

What is the goal?
Which constraints are non-negotiable?
Which are preferences?
What information and permissions does the Agent have?
What plan will it take?
Which decisions can it make autonomously?
Which actions require approval?
How do we determine that the task is complete?
How can the user modify, pause, take over, or revoke the work?
Who bears responsibility for failure and loss?

The core exchange on the Human–Agent edge is therefore not text, but intent, judgment, and agency.

This edge does not have to look like a chat GUI. It can be voice, a structured plan, a dynamic form, a canvas, a notification, object manipulation, or a combination of media. Natural language is good at expressing ambiguous goals, context, and exceptions, but poor at extensive comparison, precise parameter tuning, complex diff review, and continuous state monitoring. A more common future may use natural language for intent and structured interfaces for plans, evidence, approvals, and control.

Agent ↔ System: The Perception and Action Interface

The Agent must turn an abstract human goal into concrete actions in a real system. This edge may be called the Agent–Computer Interface (ACI), Agent–System Interface, or, more broadly, an Agent-facing Interface.

The Agent receives from the system:

current state and available objects;
data, events, and context;
capabilities, permissions, and limits;
operation results, progress, and errors.

The Agent sends to the system:

queries and commands;
tool calls and API requests;
changes to files, data, and system state;
transactions, retries, rollbacks, and compensating operations.

The core exchange on this edge is observation and action.

CLI, APIs, function calls, and MCP can all form an ACI, but none of them is the ACI itself. An Agent can also use an existing GUI through the DOM, accessibility tree, screenshots, coordinates, and keyboard and mouse actions. More precisely, the Agent–System Interface is anything that lets an Agent observe system state and act on the system.

Different media have different properties:

API / MCP / typed tools: explicit semantics and structure, usually more reliable;
CLI / Shell / code execution: general, composable, and naturally leaves a readable action record;
DOM / GUI / computer use: covers systems without structured interfaces, but is more vulnerable to visual ambiguity and interface changes.

CLI takes on a different position here than it did in traditional human-computer interaction. People once used CLI to operate the system directly, learning its language and translating goals into commands themselves. For an Agent, CLI can be an action interface: the Agent understands the goal, chooses commands, organizes parameters, and handles returned results, while CLI exposes system capabilities as callable, composable, recordable operations.

Before
Human → CLI → System
 
With an Agent
Human → Agent → CLI / API / MCP → System

This is not CLI replacing GUI again. The same interface is serving a new actor. The memory, composition, and execution work once demanded from people becomes useful material for Agents; the expressive power, composability, and textual record once offered to people become the basis for Agent action, automation, and auditing.

A poor Agent Interface merely translates a person’s page actions into tools:

click_create_button
select_region_dropdown
fill_instance_name
click_confirm

A better Agent Interface organizes capabilities around domain intent:

search_deployment_options
preview_deployment
create_deployment
get_deployment_status
rollback_deployment

The former asks the Agent to simulate clicks in a GUI. The latter lets it operate directly on the system’s domain model.

This is closely related to traditional UI design. Good UI does not mechanically expose database fields, and a good Agent Interface should not mechanically wrap backend endpoints. Both reorganize system capabilities around the user’s task model. The difference is that the new “user” is probabilistic, context-limited, and capable of misunderstanding tool semantics.

Human ↔ System: The Verification and Takeover Interface

Agents will not make traditional GUI disappear, but they will change its role.

As Agents take on more execution work, GUI will partly shift from “the panel where the user completes every operation” to “the control surface where the user supervises and governs an actor”:

view the system’s real state;
inspect changes and diffs produced by the Agent;
compare options, costs, and consequences;
approve high-risk actions;
handle exceptions and conflicts;
interrupt the Agent or take over manually;
undo, roll back, and recover;
inspect sources, evidence, and audit records.

The complexity carried by GUI changes accordingly. It used to help people perform operations; now it must also help them understand and govern another actor. Once operation steps become fewer, plans, permissions, impact, uncertainty, and execution evidence need to become clearer.

If an Agent says “the deployment is complete,” the user should not see only a natural-language conclusion. An independent fact channel should also show:

Deployment status: successful
Resources actually created: 3
Estimated monthly cost: ¥463
Public IP: none
Operator: Agent A
Permission used: deployment.write
Rollback window: 24 hours

“Human in the loop” does not mean putting a confirmation dialog before every step. Without the plan, parameters, impact, current state, and alternatives, the user lacks the information needed for meaningful judgment. Real human control requires people to know what is about to happen, what actually happened, and how to recover from deviation.

The core exchange on this edge is evidence and control.

The Center of the Triangle: Shared Tasks, State, and Authority

Designing the three edges separately is still not enough. Human, Agent, and System must collaborate around the same task model and system state.

The center of the triangle needs a shared layer:

Shared Task / State / Authority Model shared task model, shared system state, and records of permissions and responsibility.

This shared layer is not simply context that Human and Agent see at the same time. Nor does it require both sides to receive exactly the same information. For an Agent, context is the current accessible working set; it may be selected, compressed, or stale. For a Human, the interface presents a view organized for understanding, judgment, and control. Their forms differ, but they must point to the same objects, versions, plans, and action records.

The shared layer is therefore closer to an authoritative collaboration state. It is the common factual basis beneath Human and Agent context, and the basis for determining whether a plan, approval, or permission is still valid.

It must ensure that:

the objects and versions the user sees are the objects and versions the Agent is operating on;
after the Agent calls a tool, the GUI reflects real system changes;
after the user changes the system directly, the Agent can detect the new state;
approvals are bound to a specific plan, parameters, and system version;
the Agent cannot use an old authorization to perform a new action after the underlying state changes;
every action can explain who performed it, why it was performed, what changed, and where its evidence came from;
Human and Agent can continue or recover from the same checkpoint.

For example, an Agent submits this deletion plan:

Delete Project A
including 3 services and 1 database
estimated savings: ¥700 / month

After the user approves it, suppose a new production database is added to Project A. The old approval must not automatically cover the new state. The plan is stale; the system must preview the impact again and request approval.

These problems are neither only UX problems nor only MCP schema problems. They are problems of state consistency, temporal consistency, and permission consistency across the three edges.

The Real Design Object Is Agency

The three participants in an Agent system are not symmetrical.

Human is the subject of goals and responsibility;
Agent is a delegate granted limited power;
System is the environment where state, resources, and real consequences exist.

Agency is not a one-time switch that transfers power from Human to Agent. Perception, proposal, selection, execution, approval, interruption, and rollback can belong to different actors, and can be redistributed as the task stage, risk, and system state change. An Agent may be allowed to search and compare but not purchase; create a draft but not publish it; execute an approved plan but not reuse that approval after the object or parameters change.

The three edges run at different frequencies. Agent–System is a continuous, high-frequency execution loop. Human–Agent is a medium-frequency supervision loop organized around plans, progress, and exceptions. Human–System is a relatively low-frequency but indispensable loop for verification, takeover, and recovery. Different frequency does not mean different importance. The low-frequency safety edge often determines whether a system remains controllable during an exception.

The hardest design problem may not be the interface of each edge, but how control flows among all three:

When must the Agent ask the Human?
Which risks are worth interrupting the user for?
How much action can one approval cover?
After the Human changes the system, how does the Agent learn that its context has changed?
After failure, how can a person continue from an intermediate state?
After a person takes over, how can the task be handed back to the Agent?
When multiple Agents act at once, who has the latest valid state?
When an Agent switches from API to GUI, do permissions and audit trails remain consistent?

The design material of the Agent era therefore expands from pages, components, and flows to:

Goal / Context: how goals are expressed and what each party knows;
Capability / Permission: what an Agent can do and which actions require authorization;
Plan / Approval: how work is decomposed and when human judgment is needed;
Agency / Control: who may judge, act, interrupt, and take over;
Uncertainty / Evidence: how uncertainty is shown and how conclusions are proved;
Trace / Recovery: how the process is observed and how it recovers after failure;
Evaluation / Responsibility: how collaboration is measured and who bears the consequences.

The Missing Gap

Products and organizations still usually separate the three edges.

Traditional design teams mainly own the Human ↔ System GUI. Conversation design and AI product teams own Human ↔ Agent. Platform and engineering teams own the Agent ↔ System APIs, CLIs, and MCPs. Each group may locally optimize its own interface, while no one is responsible for the quality of collaboration across the entire triangle.

The results include:

an Agent’s plan does not correspond to the real objects in the GUI;
the user approves a sentence-long summary rather than definite actions and parameters;
an MCP call changes the system while the supervision surface fails to update;
after manual takeover, the Agent continues from stale context;
the Agent can perform an action but cannot provide enough evidence;
the system records technical logs but cannot explain goals, decisions, or responsibility;
the product pursues higher automation while ignoring irreversible errors and the cost of user correction.

This gap cannot be filled simply by adding an AI designer role. Product, design, and engineering must face the same domain model, state model, permission model, and evaluation standards.

Agent products also cannot be evaluated only through traditional task completion, elapsed time, and satisfaction. We must also ask:

How often does the Agent choose the correct tool?
How closely does its plan match the user’s actual intent?
How frequently do invalid calls and invalid approvals occur?
What does it cost the user to discover and correct a deviation?
Can the Agent request help at the right time?
What are the recovery rate and irreversible-error rate?
How often do stale context, stale plans, and stale permissions fail?
Does user trust in the Agent’s capability match its actual reliability?

Redefining Interface Design

The future is not UI being replaced by Agent Experience (AX), nor GUI being replaced by CLI or MCP.

The more likely combination is:

Natural language expresses intent; Agents plan and execute; GUI presents state, comparison, evidence, and control; CLI / API / MCP connect to the real system.

From CLI to GUI and then to Agents, complexity moves from human memory and operation to system representation and constraint, and then to delegated planning and execution. The human role shifts toward setting goals, authorizing, supervising, verifying, and taking over. The design object expands from an individual page and a single operation to a complete loop spanning goals, plans, actions, evidence, approvals, takeover, and recovery.

The triangle can be summarized in three sentences:

Human ↔ Agent: design how to delegate. Agent ↔ System: design how to act. Human ↔ System: design how to verify and take over.

In most products of the past, UI Design primarily designed how people operated systems.

Interface Design in the Agent era designs how people delegate limited agency, how Agents act on systems, and how systems prove to people what actually happened after some complexity has been delegated.

Ultimately, we are no longer designing only what an interface looks like, or even the experience of one actor. We are designing an agency architecture.

Agency here does not mean unconstrained autonomy. It means the capacity to perceive, judge, and act within concrete goals, permissions, and boundaries of responsibility.

Who can perceive, who is responsible for judgment, who is authorized, who actually acts, who verifies the result, who can interrupt or revoke the action, and who bears final responsibility.

That may be the true starting point for rethinking the interface.