About
    • AMMDI is an open-notebook hypertext writing experiment, authored by Mike Travers aka mtraven. It's a work in progress and some parts are more polished than others. Comments welcome! More.
Search
Full

The AI/UI Thesis

15 Oct 2026 - 15 Oct 2026
Open in Logseq
    • The AI/UI thesis blyg
      • An expansion of the AI/UI thesis outline, with AskClaude sub-bullets added under each item. Note: apologies if all this is pretty obvious; some of us are still catching up.
      • No matter how powerful AI agents get, there will always be a need
        • Ask Claude
          • The claim is strongest when stated as a principal–agent problem rather than a UI problem. The agent's capability is not the bottleneck. The bottleneck is the bandwidth and fidelity of the channel carrying what you want and what it's doing. Delegation among humans never eliminated managers, specs, or status reports.
          • Counterargument worth taking seriously: a capable enough agent infers intent from context (your files, history, past corrections), so explicit expression shrinks. The rebuttal is that inference creates a new UI need, which is inspecting and correcting the agent's model of you.
        • for better ways to express intent to them Ask Claude
          • The current default is the chat box plus ad-hoc artifacts (CLAUDE.md files, skills, specs, plan mode). These are all text, mostly unstructured, and poorly versioned.
          • Underused intermediate forms that sit between natural language and code:
            • editable plans
            • constraints and invariants ("never touch X")
            • examples and counterexamples
            • partial programs with holes
            • direct manipulation of a draft output
          • Intent is not a single message. It is a persistent, evolving object. It should be a first-class, inspectable, diffable artifact rather than scattered across chat history.
          • Granularity matters. People want to give goals at one level and be consulted at another. An interface for setting the "consult me when…" threshold is mostly missing.
        • tools for monitoring and interacting with them Ask Claude
          • Monitoring needs to work at multiple zoom levels:
            • portfolio: which of N agents needs me?
            • trajectory: what is this one doing, and why?
            • action: what exactly did it change?
          • Interaction modes beyond approve/deny:
            • interrupt
            • redirect mid-task
            • take over and hand back
            • fork
            • annotate for future runs
          • The hard problem is calibrated attention. Approval prompts at every step cause rubber-stamping. Fully autonomous runs hide errors. The interface has to route human attention to where it matters.
      • This is an underexplored area
        • Ask Claude
          • It is partly true. It is underexplored as a design discipline, but it is now crowded as a product category. HCI research is catching up. ACM IUI 2026 ran a workshop (AgentCraft) on interfaces for meaningful oversight of agentic plans, decisions, and actions, framed around developers struggling to understand and debug agent behavior and to know when human oversight is needed.
          • What is still underexplored:
            • representations of agent intent and state that aren't transcripts
            • intent as a persistent structure
            • interfaces for non-programmers that are more than chat
            • anything drawing on the visual programming and end-user programming literature
        • The more capable AI gets, the more this interface becomes the bottleneck and improving it provides value Ask Claude
          • This is Amdahl's law applied to human–agent systems. As agent time per task falls, human specification and review time dominates.
          • Industry is already noticing. One vendor essay argues that enterprise UIs were built for human task execution, and that what's needed is a trust layer designed for oversight rather than execution.
          • Caveat: the value accrues to whoever owns the agent loop (Anthropic, OpenAI, Cursor). An independent UI layer needs either open protocols to attach to or a niche the platforms won't serve.
      • My dissertation (1996) actually addressed this problem, tangentially. Ahead of my time! Ask Claude
        • The direct link is anthropomorphic, goal-driven agents as a programming metaphor, with agents made visible in the environment (LiveWorld). That is the same problem as today's legibility problem: making autonomous processes comprehensible to the human who set them in motion.
        • Worth extracting concretely:
          • How did agents represent goals?
          • How were conflicts between agents shown?
          • What did "watching an agent" look like?
          • These map directly onto intent expression and monitoring.
        • This could be a useful framing piece for publication: "what 1990s agent-programming research got right about LLM agents." It gives the thesis a distinctive angle that most current builders lack.
      • What is the current state-of-the-art?
        • I'm not sure and need to research it. There must be dozens of agent harness type things that do stuff in this ballpark and I'm not that familiar with them Ask Claude
          • Here is a rough map as of Oct 2026.
          • **Protocols (the plumbing layer)**
            • MCP handles tools. A2A handles agent-to-agent communication.
            • AG-UI is an event-driven protocol (over SSE or WebSockets) for streaming reasoning steps, syncing state, and rendering dynamic UI components between agent backends and frontends.
            • Generative-UI specs ride on top. A2UI defines what UI to render, and AG-UI defines how agent and UI talk in real time.
            • Consolidation is underway. Thoughtworks notes newer MCP apps package HTML/UI widgets directly in MCP servers or skills, putting a separate UI protocol layer in question.
          • **Multi-agent dashboards (the monitoring layer)**
            • This is now table stakes for coding agents. Claude Code launched Agent View on May 11, 2026: one dashboard of background sessions that flags which agents are waiting on input.
            • Codex CLI added an equivalent agents dashboard plus a queue command for messaging sessions from outside in August 2026.
            • Grok Build added a similar dashboard in June 2026.
            • Open-source tools exist too (Stoneforge, Emdash, PolyPilot, and others), mostly kanban-of-sessions over git worktrees.
            • Common gap: most still leave task alignment, conflict resolution, and merge decisions to the human.
          • **Research prototypes (the interaction-design layer)**
            • Microsoft's Magentic-UI treats the human as a special agent in the team and offers six mechanisms: co-planning, co-tasking, action approval, answer verification, memory, and multi-tasking.
            • This is the closest thing to a design vocabulary for the field.
          • Assessment
            • Monitoring has converged on "session list with status badges." That is shallow.
            • Intent expression has barely moved past chat plus markdown config files. That is the bigger opening.
            • Almost nothing exists outside coding.
          • Next step: spend a few hours hands-on with Magentic-UI, one orchestrator, and AG-UI/CopilotKit rather than reading more.
      • I should just build one (see PAgents KnowOS etc ) Ask Claude
        • Don't build "a harness." That market is saturated and owned by the model vendors. Build the piece they won't: a richer intent/monitoring layer that attaches to existing harnesses via MCP, MCP apps, or hooks.
        • Your advantages:
          • a Clojure/data-centric stack (intent as data, Datomic-style history of agent actions)
          • a visual-programming background
          • a real domain: scientific data workflows at Parker, where oversight genuinely matters and generic coding dashboards don't fit
        • Pick one concrete user and task, for example "an agent doing scRNA-seq QC and analysis, with a human steering." Build the UI for that. Generalize afterward.
        • (I don't have context on what PAgents and KnowOS currently contain. Worth a line each here on what they do and why they stalled or didn't.)
      • It's difficult to know what to build given how fast things are changing Ask Claude
        • Ask what is stable underneath the churn. Models, harnesses, and protocols will turn over. Some things won't:
          • the need for durable representations of intent, history, and provenance
          • the need for human attention routing
          • domain data models
          • Build on those; wrap the volatile parts behind thin adapters.
        • Bet on whatever survives models getting 10× better. Monitoring at the action level may fade. Intent specification and outcome verification won't.
        • Ship small, public pieces fast. The landscape rewards being visible early more than being architecturally complete.
      • One obvious approach – build a lot of small, composable components. I'm trying to do this with Hyperphor stack. But I need to rethink in depth (that's what this is for.) Ask Claude
        • Composability is right, but it needs a unifying substrate or it becomes a pile of parts. A candidate substrate: a shared, queryable, append-only log of intents, plans, agent actions, and human interventions. Every component reads or writes it.
        • Make components agent-consumable as well as human-consumable:
          • each one exposed as an MCP tool or an MCP app
          • the agent can render the UI
          • the human can drive the same component directly
        • Watch for the familiar risk: infrastructure that defers shippable output. Set a rule that each component must be demoable in an end-to-end task within weeks.
      • TODO take every bullet item above and sic claude on it, with context Ask Claude
        • Done above. Weakest spots, which need your input:
          • what PAgents, KnowOS, and the Hyperphor stack currently consist of
          • which specific dissertation ideas you think transfer
        • Sources: