Agent Workbench: Problems and Reflections from Building an Agent Project

An AI agent workbench built with Bun and TypeScript, supporting multiple models, multiple transports, and a pluggable tool system.

1. Why Build It?

After reading plenty of documentation and examples on AI application development, I had gained a fairly broad understanding of how agents work—from ReAct to Toolformer, and from MCP to Agent Skills. But there is a huge gap between understanding what you read and knowing how to build it.

The best way to learn is to build one yourself.

So I decided to start from scratch, beginning with the protocol layer and gradually building my own agent workbench. The aim wasn't to replace Cursor or Claude Code, but to understand the engineering decisions beneath an agent system by building each module myself: protocol design, task scheduling, tool registration, and event streams.

Before starting, I also studied several existing agent frameworks. I'll use these excellent projects as references while building my own general-purpose agent.

The project's core design goals are:

Model independence: no ties to a particular AI vendor; switch freely between OpenAI, Anthropic, DeepSeek, and others through configuration.

Transport independence: the kernel doesn't care where messages come from. Stdio, Electron IPC, and a possible future WebSocket transport can all connect.

Pluggable tools: built-in filesystem, shell, and task-management tools, plus support for external tools through MCP (Model Context Protocol).

Multi-agent collaboration: the main agent decides whether to delegate subtasks. It handles simple tasks itself and distributes complex ones, scheduling flexibly and parallelizing when needed.

Cascading configuration: global → workspace → session, with three layers of priority-based overrides for flexibility and control.

Platform independence: abstracting the UI and communication protocol (JSON-RPC) lets the frontend run on any platform supporting web technologies, including Electron and browsers. Core logic connects through the same protocol contract. Inspired by projects such as OpenClaw, this makes the kernel a real backend service rather than an embedded component tied to a particular shell.

2. Technology Choices

Why Bun Rather Than Node.js?

Bun's isolated linker avoids traditional Node "node_modules hell," and workspace packages in the monorepo can be referenced immediately without a build. Its test runner also offers excellent Jest compatibility and execution speed. Most importantly, in the Electron desktop app, Bun runs the core logic as a subprocess with almost no startup delay.

Why Vercel AI SDK Instead of Calling APIs Directly?

Vercel AI SDK has been widely validated in Anthropic's Agent Skills documentation and community practice. Its unified streamText / generateText interfaces and ToolLoopAgent abstraction avoid writing repetitive adapters for every model provider. I only use it for underlying model calls and tool loops, though; routing, configuration, and task scheduling remain under my own control.

3. Project Design

3.1 Architecture Evolution: From Blindly Pursuing Multiple Agents to a Configurable Agent System

Task execution architecture is where I spent the most time thinking—and encountered the most pitfalls.

First Pitfall: Blindly Pursuing Multiple Agents

Like many people, I initially thought "multi-agent collaboration" sounded impressive: the main agent plans, subagents execute, and everyone has a role. So I built the typical Main Agent + SubAgent arrangement:

Once it ran, the problem quickly became obvious: a huge number of tokens went into synchronizing context between agents. Subagent A wrote code and returned it to the main agent, which passed it to subagent B for tests. After B finished, the main agent integrated the result and sent it to subagent C for review. Every handoff consumed substantial tokens. Information also degraded at each step of this relay, and the final results were disappointing.

Building multi-agent systems describes this problem well: dividing agents by task type—planner, implementer, tester, reviewer—is the most common mistake. Such pipelines lose context at every handoff, and subagents spend more tokens coordinating than doing useful work. The article gives a direct estimate: multi-agent systems typically consume 3–10 times as many tokens as a single agent. The better approach is context-centric decomposition. An agent that already understands a feature's implementation should also write its tests, rather than handing them to another agent unfamiliar with the details. Multiple agents are worth introducing only when their contexts can truly be isolated, such as parallel searches across different sources or changes to independent modules with clear interfaces.

Returning to One Agent: Much Better Quality, but New Problems

After that experience, I returned to the simplest approach: one agent + tools + skills. The results actually improved dramatically. Without the loss from passing context around, the agent could complete the whole task coherently.

But two new problems emerged:

1. Homogeneous Output

Within one session, the agent has to write code, tests, and documentation. Without a role-switching mechanism, these outputs converge in style and reasoning. This isn't beneficial consistency; it is mental inertia. Tests validate the code using the same assumptions that produced it, while documentation tells the story from the same angle. Independent perspectives are missing.

2. Limits of a Single Agent

One agent clearly struggles in some situations:

Large-scale search: when searches need to span multiple repositories or documentation sources in parallel, a single agent searches sequentially, with time growing linearly with scope.

Parallel changes across modules: when several independent modules need changing, one agent cannot advance them in parallel and must switch context repeatedly.

Degraded tool selection with too many tools: once the tool count grows beyond about 15–20, the model starts selecting tools incorrectly. Adding tools for one domain can even hurt performance in another.

These problems correspond to the three scenarios where Anthropic's multi-agent article says multiple agents are useful: parallelization for large searches and independent modules, context isolation to avoid mental inertia, and specialization to separate tools by responsibility and prevent overload.

The Final Approach: A Configurable Agent System

I didn't hard-code a "main agent → split task → delegate to subagents" workflow. Instead, I designed a mechanism that makes each agent a configurable unit:

The central idea is simple: whether an agent can launch subagents depends on whether it has the delegate_task tool. Give an Orchestrator this tool and an appropriate system prompt. Omit it for a Worker that only executes tasks. Every agent type—Orchestrator, Explorer, Worker—is defined through configuration rather than hard-coded.

The flexibility shows up in several ways:

New agent types require no code changes: want a Code Reviewer Agent? Configure a system prompt and read-only tools. No implementation changes are needed.

Agents are decoupled from models: an Orchestrator can use Opus for complex decisions, a Worker can use Sonnet for efficient execution, and an Explorer can use Haiku for quick searches. Choose models by responsibility and allocate them as needed.

Tools define capability boundaries: what an agent can and cannot do is determined entirely by its tools. delegate_task is just another tool, not fundamentally different from the others.

Simply put: when one agent is enough, it works efficiently on its own. When multiple agents are needed, configure an Orchestrator with delegate_task, and it decides when to split the task and whom to delegate to.

3.2 Design Philosophy: Protocol First

The whole project follows a protocol-first philosophy. Before writing any kernel code, define the communication contracts between layers.

The center of that contract is packages/protocol, which defines:

15 JSON-RPC methods covering configuration (config.get/set), workspace CRUD (workspace.create/list/set/delete), sessions (session.create/delete/list/set/detail), task control (task.create/cancel), and system capabilities (kernel.capabilities).

6 server-pushed events for configuration changes (app.config.changed) and the task lifecycle (task.started/delta/succeeded/failed/cancelled).

Complete Zod validation: strict runtime checks for every request's inputs and outputs, and for event data.

With the protocol as the single source of truth, the SDK and kernel can evolve independently. As long as the protocol remains compatible, changes on either side don't affect the other. This also makes the kernel transport-agnostic: it processes protocol-compliant messages without caring whether they arrive through stdio or WebSocket.

3.3 Cascading Configuration: Three Layers of Overrides

A core issue for AI agents is context configuration. LangChain's Context Engineering for Agents emphasizes that context includes not only prompt text but also runtime settings such as model choice, tools, and skills.

I designed three layers of cascading configuration:

Each layer is a RuntimeConfig defining modelId, agentId, skillPaths, toolTitles, and mcpNames. Fields absent from a lower layer fall back to the layer above, ultimately reaching the agent's preset defaults. This allows:

Global defaults for models and tools.

Workspace overrides for specific projects, such as pinning one project to Claude Opus.

Session overrides that temporarily enable or disable tools or skills.

Looking back at the three elements in Lilian Weng's LLM Powered Autonomous Agents—planning, memory, and tool use—my configuration system essentially makes those elements configurable at runtime.

3.4 Reducing Boilerplate: A Minimal IoC Container

In practice, modules frequently need shared context objects: loggers, configuration, database connections, and so on. Passing them manually adds parameters to every function and quickly creates boilerplate across layers. I didn't want a heavyweight DI framework such as InversifyJS or TSyringe for this small project, so I wrote an IoC container of roughly 50 lines using AsyncLocalStorage:

AsyncLocalStorage automatically carries a container context with each request. Callers don't pass it explicitly: inject(LoggerToken) in any function retrieves the current request's Logger, and separate requests don't interfere with each other's contexts. inject.lazy() uses a Proxy to resolve a dependency only on first property access, addressing circular dependencies between providers.

3.5 The Tool System: Built-In Tools and MCP

Tools are an agent's hands. The project includes 14 tools—reading, writing, editing, and deleting files; directory listing; glob and text search; patch application; shell execution; and task creation, updates, listing, and delegation—covering the core needs of a coding agent.

External tools also connect through the standardized Model Context Protocol (MCP). McpManager supports three transports:

HTTP: remote MCP services.

SSE: Server-Sent Events.

Stdio: local process communication, such as connecting a local LSP server.

4. Overall Architecture

4.1 Process Model

Key design decisions:

The kernel is a pure logic layer: it contains no transport-specific code. It accepts Request objects, returns Response objects, and broadcasts Event objects through callbacks. That lets it run unchanged inside a stdio process, an Electron main process, or a possible future HTTP server.

The host is the process boundary: it starts the transport, loads the kernel, and validates requests. As the kernel's runtime environment, it owns all process-related responsibilities.

The desktop shell is the UI layer: the Electron renderer handles presentation, while the main process bridges IPC and stdio, forwarding renderer requests to the host process.

4.2 Data Flow

A typical conversation request follows this path:

Each step has a clear responsibility, and layers connect through type-safe contracts.

4.3 Context Engineering in Practice

Prompt templates: prompt fields in runtime configuration can be customized by layer. The agent's system prompt is dynamically assembled from the prompt, skill list, and tool list.

Message context management: ContextProvider collects messages from peer tasks in a session and sorts them chronologically into a complete conversation history.

Tool results become context: tool inputs and outputs are stored as structured function_call parts, allowing the agent to review its own action history.

5. Project Structure

6. Core Modules in Detail

6.1 Kernel

This is the system's brain, with bootstrap() as its main entry point:

bootstrap() exposes only handleRequest. Routing, business execution, and error handling happen transparently inside it. This pipeline design makes the kernel's implementation a complete black box to outside callers.

The kernel broadcasts events through a callback registered under EmitToken. It doesn't need to know who consumes the events, keeping it fully transport-agnostic.

6.2 Protocol

The protocol layer is the project's most important design decision. Every cross-process or cross-package message undergoes strict type validation:

Every cross-process message is checked by ReqSchema.safeParse(). Invalid requests are rejected before they enter the kernel, so they cannot corrupt internal state.

6.3 SDK: The Client Abstraction

ClientSdk is the UI's only interface. It correlates requests and responses by request ID, dispatches server events, and manages the transport lifecycle:

This design shields the UI from transport details. Stdin/stdout, Electron IPC, and a possible future WebSocket transport are all transparent to it.

6.4 UI: The Workbench Interface

The UI uses React 19, Zustand, and react-router-dom 7. Its central design is event-driven state reduction:

As a React Context Provider, WorkbenchControllerProvider does three things on mount:

Fetch initial data: workspaces, sessions, and a configuration snapshot.

Subscribe to kernel events: listen for app.config.changed to update global configuration and task.* events to update active task state.

Reduce task deltas: turn streaming task.delta events into a structured list of TaskParts.

The workbench has a sidebar with sessions and drag-and-drop project creation, and a main panel with messages and an input area. Settings provide visual model/provider management, MCP server switches, and tool/skill toggles. All changes persist through the config.set RPC.

6.5 Desktop: The Electron Shell

Traditionally, core logic is bundled into Electron's main process. Bun fits naturally as a subprocess instead: start it with child_process.spawn("bun", ["run", "host"]) and communicate over stdio using JSON-RPC. This also leaves room for other client frameworks later:

Kernel logic is fully separated and can be tested and deployed independently.

A main-process crash doesn't affect the kernel.

CLI and desktop shells can run on the same host and share the same kernel implementation.

7. Summary and Outlook

There is still plenty to improve, but the overall framework is in place. Agent Workbench is my first complete engineering exercise in building an AI agent workbench from the protocol layer to the UI. The project is available at agent-workbench.

Next steps include:

Filesystem tool implementation: protocols for read/write/edit/delete/search and other tools are defined; their core logic is being implemented.

Plugin system: configure MCP services and accompanying skills in one step, making tools and knowledge available out of the box.

More transports: add WebSocket and HTTP alongside stdio and Electron IPC to support remote execution.

Worktree isolation: run each session in its own Git worktree to avoid file conflicts between concurrent tasks.

A richer skill ecosystem: support skill dependencies and integrate mainstream skill marketplaces.