Background tools and sub-agents

[tcdent]

Summary

Adds support for running tools in the background with a background: true parameter, spawning sub-agents for parallel research tasks, and fixes critical concurrency bugs.

Features

Background Tool Execution

Any tool can now be run in the background by passing background: true. The tool returns immediately with a task_id, allowing the conversation to continue while work happens in parallel.

User: Run these three commands in parallel
Assistant: [calls mcp_shell with background: true for each]

The tasks are running. You can check status with list_background_tasks 
or retrieve results with get_background_task(task_id).

Sub-Agents (mcp_task)

New mcp_task tool spawns a background sub-agent to handle research and analysis tasks:

• Has read-only tool access (read_file, shell, fetch_url, web_search)
• Runs independently while main conversation continues
• Results retrieved via get_background_task

Task Management Tools

• list_background_tasks - Shows all running/completed background tasks
• get_background_task(task_id) - Retrieves result and removes from tracking

Both have custom UI rendering and support auto-approve via config:

[tools.list_background_tasks]
allow = [".*"]

[tools.get_background_task]
allow = [".*"]

Bug Fixes

Background Tool Output Loss

Problem: When multiple background tools ran concurrently, some would lose their output entirely.

Root cause: The main select! loop could drop futures mid-await, losing handler results stored in local variables.

Fix: Spawn handlers in separate tokio tasks. Results sent via oneshot channel and polled in poll_waiting(). Spawned tasks run to completion regardless of executor state.

Concurrent Tool Approval

Problem: When Claude called multiple tools at once, only the first showed the y/n approval prompt.

Fix: Added pending_approvals queue in app.rs to serialize approvals. Each tool gets its turn for the approval dialog.

Busy-Polling While Waiting for Approval

Problem: Tight loop when waiting for user input, spamming logs and wasting CPU.

Fix: 10ms sleep when blocked on approval/effects. Reduced poll logs to trace level.

Architecture Changes

WaitingFor State Machine

Replaced separate Option fields with enum:

enum WaitingFor {
    Nothing,
    Handler(oneshot::Receiver<Step>),  // Spawned handler execution
    Effect(oneshot::Receiver<EffectResult>),  // Delegated to app layer
    Approval(oneshot::Receiver<ToolDecision>),  // User input
}

Tool Block Status

• Removed unused Status::Queued variant
• Blocks now marked complete by call_id instead of "active block" index

Documentation

Added comprehensive src/tools/README.md covering:

• Architecture overview with diagrams
• Step-by-step guide for adding new tools
• Pipeline patterns (basic, with approval, with cleanup, delegation)
• Testing examples

Testing

• 5 new unit tests for concurrent background execution
• Manual testing verified 6+ concurrent tasks (shell, read_file, fetch_url, web_search, task) all complete with output preserved

Stats

24 files changed, 2188 insertions(+), 319 deletions(-)