Running the Examples

The framework includes full-featured demos for both Gemini and OpenAI, with multiple agents, tools, image generation, and a web client with audio playback.

Prerequisites

• Node.js 22+
• A Google API key with Gemini Live API access, or an OpenAI API key
• Chrome (recommended for the web client)

Start the Agent Server (Gemini)

# Set your API key
export GEMINI_API_KEY="your-key-here"

# Start the voice agent
pnpm tsx examples/hello_world/agent.ts

You should see:

============================================================
Bodhi Realtime Agent — Gemini Voice Assistant
============================================================

  WebSocket audio server: ws://localhost:9900
  Session ID: session_1234567890

Connect a WebSocket audio client and try saying:
  - 'What time is it?'
  - 'What is 25 times 17?'
  - 'I need help with complex math' (transfers to math expert)
  - 'What's the weather in San Francisco?' (uses Google Search)
  - 'Use slow search for AI news'
  - 'Speak slower please' (changes speech speed)
  - 'Generate an image of a sunset' (creates and displays image)
  - Try speaking in any language — the assistant replies in kind

Press Ctrl+C to stop.
============================================================

Start the Agent Server (OpenAI)

Alternatively, run a voice assistant on OpenAI's Realtime API:

# Set your API keys (Gemini powers the image/video subagents)
export OPENAI_API_KEY="your-key-here"
export GEMINI_API_KEY="your-gemini-key"

# Start the voice agent
pnpm tsx examples/hello_world/openai-realtime-tools.ts

The OpenAI example is a senior-friendly assistant: calculator, current time, speech speed, image and video generation, and a transfer to a patient math helper. The web client works with either server without changes — audio format is negotiated automatically via session.config.

Start the Web Client

In a second terminal:

pnpm web-client

Open http://localhost:8080 in Chrome and click Connect.

Things to Try

Say this	What happens
"What time is it?"	Calls get_current_time tool, speaks the result
"What is 25 times 17?"	Calls calculate tool with the expression
"I need help with complex math"	Transfers to the math_expert agent
"Transfer me back" (with math expert)	Returns to the main agent
"What's the weather in Tokyo?"	Uses Google Search grounding for real-time data
"Generate an image of a sunset"	Calls Imagen API, image appears in browser
"Speak slower please"	Adjusts playback rate via set_speech_speed tool
"Use slow search for AI news"	Demonstrates 3-second slow tool (agent keeps talking)
Speak in any language	The assistant automatically replies in the same language

Agents in the Demo

Main Assistant

The default agent with access to all tools. Handles general conversation and routes specialized requests to other agents.

Math Expert

A specialist with a professorial tone. Activated when you ask for help with complex math. Has the calculator tool and can transfer back to main.

Multilingual

The main assistant is multilingual by default — speak in any language and it replies in kind. No separate agent needed; Gemini's native audio model handles language detection and response automatically.

Environment Variables

Variable	Default	Description
GEMINI_API_KEY	—	Google AI Studio API key (for Gemini example)
OPENAI_API_KEY	—	OpenAI API key (for OpenAI example)
PORT	9900	WebSocket port for the agent server
CLIENT_PORT	8080	HTTP port for the web client
WS_URL	ws://localhost:9900	Agent WebSocket URL (used by web client)

Troubleshooting

No audio playback

Make sure you click the Connect button directly — Chrome requires a user gesture to enable audio. Check the debug log for playChunk error messages.

No user transcription

User speech transcription uses a two-layer approach:

1. Chrome STT (client-side) — the browser's SpeechRecognition API provides real-time interim text during speech. Only works in Chrome with a working microphone.
2. Server STT (server-side) — GeminiBatchSTTProvider produces the authoritative transcript after each turn, replacing the Chrome STT text.

If you see no transcription at all, check that you're using Chrome and that microphone access is granted. The server STT transcript appears after the model starts responding (not during speech).

Agent doesn't call tools

Check the server terminal for [Hook] Tool called: messages. If no tool calls appear, try being more explicit: "Use the calculator to compute 25 times 17" instead of just "25 times 17".

Transfer not triggering

The model must invoke the transfer_to_agent function call, not just say it verbally. If transfers aren't happening, check the server logs for [Hook] Agent transfer:.

Image generation fails

Image generation uses the Imagen API (imagen-3.0-generate-002). Some API keys may not have access. Check the server logs for [Tool] Imagen failed messages.

Claude Code Demo — Voice-Driven Coding

A voice assistant backed by Claude Code (Anthropic's AI coding agent). This demo uses the relay subagent pattern — an interactive subagent bridges the voice model to Claude Code's stateful SDK, managing session lifecycles and question relay.

Prerequisites

• A Google API key (Gemini Live API)
• An Anthropic API key (Claude Agent SDK)
• macOS (for Apple Mail email integration)

Start the Agent Server

export GEMINI_API_KEY="your-gemini-key"
export ANTHROPIC_API_KEY="your-anthropic-key"
export PROJECT_DIR="/path/to/your/project"  # optional, defaults to cwd

pnpm tsx examples/claude_code/claude-demo.ts

You should see:

============================================================
Bodhi + Claude Code — Voice-Driven Coding Assistant
============================================================

  Voice agent:     ws://localhost:9900
  Project dir:     /path/to/your/project
  Session ID:      session_1234567890

Start the web client in another terminal:
  pnpm web-client

Then open http://localhost:8080 and try saying:
  - 'Fix the bug in auth.py'                 (Claude Code)
  - 'Add input validation to the login form' (Claude Code)
  - 'Run the tests and fix any failures'     (Claude Code)
  - 'Email me a summary of the README'       (Email via Mail.app)
  - 'What is the weather in San Francisco?'  (Google Search)
  - 'Draw me a picture of a sunset'          (Image generation)
  - 'Goodbye'
============================================================

Start the Web Client

Same as the Gemini demo — the web client is shared:

pnpm web-client

Open http://localhost:8080 in Chrome and click Connect.

Things to Try

Say this	What happens
"Fix the bug in auth.py"	Claude reads files, edits code, runs tests
"Add input validation to the login form"	Claude creates or modifies files
"Run the tests and fix any failures"	Claude runs bash commands and iterates
"Summarize the README and email it to user@example.com"	Claude reads files and sends via Apple Mail
"What's the weather in San Francisco?"	Google Search (Gemini native)
"Draw me a picture of a sunset"	Image generation subagent
"Goodbye"	Graceful session end

How It Works

1. You speak a coding request → Gemini calls the ask_claude background tool
2. The relay subagent receives the task and calls claude_code_start
3. Claude Code runs autonomously: reads files, edits code, runs commands
4. If Claude asks a follow-up question, the relay calls ask_user → you hear the question and answer by voice
5. When Claude finishes, the relay summarizes the result → Gemini speaks it to you

Troubleshooting

Claude makes zero tool calls: Verify ANTHROPIC_API_KEY is set. Check server logs for [ClaudeCode] SDK init message: to confirm the SDK initialized correctly.

Email not sending: Requires macOS with Mail.app configured. First use triggers a system permission dialog. Check for [MCP:send_email] Tool invoked! in logs.

OpenClaw Demo — Persistent Agent Sessions

A voice assistant backed by OpenClaw, a gateway for delegating tasks to Claude with persistent session state. This demo uses the relay subagent pattern with LLM-driven session routing and concurrent task isolation.

Unlike the Claude Code demo (which creates fresh sessions per task), the OpenClaw demo maintains persistent sessions — follow-up requests are automatically routed to the correct existing session by a classifier.

Prerequisites

• A Google API key (Gemini Live API)
• An OpenClaw gateway running locally (or remotely)
• Device identity registered with the gateway (~/.bodhi/device-identity.json)

Start the Agent Server

export GEMINI_API_KEY="your-gemini-key"
export OPENCLAW_TOKEN="your-openclaw-token"
export OPENCLAW_URL="ws://127.0.0.1:18789"  # optional, this is the default

pnpm tsx examples/openclaw/openclaw-demo.ts

Start the Web Client

pnpm web-client

Open http://localhost:8080 in Chrome and click Connect.

Things to Try

Say this	What happens
"Reschedule my 3pm meeting to 4pm"	OpenClaw manages your calendar via Claude
"What's on my calendar tomorrow?"	Reads calendar (runs in parallel with other tasks)
"Email John a summary of the meeting notes"	Claude drafts and sends email
"Generate an image of a sunset"	Image generated and stored as artifact
"Email that image to me"	Artifact attached and sent via OpenClaw
"Confirm that reschedule"	Classifier routes to the existing calendar session
Ask two things at once	Tasks run concurrently with session isolation

How Session Routing Works

1. You speak a request and Gemini calls the ask_openclaw background tool
2. The session classifier (LLM call, 3s timeout) decides: continue an existing session, or create a new one
3. The routing mutex serializes concurrent routing decisions to prevent races
4. The task manager acquires a semaphore slot and optional write lock
5. The relay subagent sends the message to the correct OpenClaw session
6. Results are delivered to you via voice when the current turn completes

Key Components

• Session Registry — Tracks active sessions with recent conversation context, domain tags, and status
• Session Classifier — LLM-driven routing with stale-thread reconciliation (max 1 retry)
• Task Manager — Concurrency semaphore (max 10), per-domain write locks, thread TTL (10 min)
• Artifact Registry — In-memory image storage for cross-tool data flow (generate then email)

Environment Variables

Variable	Default	Description
GEMINI_API_KEY	—	Google Gemini API key
OPENCLAW_TOKEN	—	OpenClaw gateway auth token
OPENCLAW_URL	ws://127.0.0.1:18789	OpenClaw gateway WebSocket URL
PORT	9900	WebSocket port for the agent server
HOST	0.0.0.0	Host to bind to

Troubleshooting

Session routing sends to wrong session: Check server logs for [Classifier] messages. The classifier has a 3-second timeout — slow LLM responses fall back to creating a new session. Increase classifierCap if you have many active sessions.

"All background agents are busy": The task manager semaphore (default 10 slots) is full. Wait for tasks to complete or increase maxConcurrent.

Image attachment fails: Artifacts must be under 5 MB (gateway limit). Check that the artifact hasn't expired (30-minute TTL). Use "list artifacts" to see available artifacts.

Device not authorized: Run openclaw gateway devices approve <device-id> on the gateway to authorize your device. Local connections auto-approve.