bodhi-realtime-agent / OpenAIRealtimeTransport

Class: OpenAIRealtimeTransport

Defined in: transport/openai-realtime-transport.ts:72

LLMTransport implementation for the OpenAI Realtime API.

Uses the openai SDK's WebSocket transport (OpenAIRealtimeWS) for bidirectional audio streaming with function calling support.

Key differences from Gemini:

• In-place session updates (no reconnect for agent transfers)
• Streamed function call arguments (accumulated before dispatch)
• Client-managed interruption (truncate + cancel)
• 24kHz audio (vs Gemini's 16kHz)
• Explicit response.create required after tool results

Implements

• LLMTransport

Constructors

Constructor

new OpenAIRealtimeTransport(config): OpenAIRealtimeTransport

Defined in: transport/openai-realtime-transport.ts:141

Parameters

config

OpenAIRealtimeConfig

Returns

OpenAIRealtimeTransport

Properties

audioFormat

readonly audioFormat: AudioFormatSpec

Defined in: transport/openai-realtime-transport.ts:84

Implementation of

LLMTransport.audioFormat

---

capabilities

readonly capabilities: TransportCapabilities

Defined in: transport/openai-realtime-transport.ts:73

Static capabilities — read before connecting, used for orchestrator branching.

Implementation of

LLMTransport.capabilities

---

onAudioOutput()?

optional onAudioOutput: (base64Data) => void

Defined in: transport/openai-realtime-transport.ts:93

Parameters

base64Data

string

Returns

void

Implementation of

LLMTransport.onAudioOutput

---

onClose()?

optional onClose: (code?, reason?) => void

Defined in: transport/openai-realtime-transport.ts:102

Parameters

code?

number

reason?

string

Returns

void

Implementation of

LLMTransport.onClose

---

onError()?

optional onError: (error) => void

Defined in: transport/openai-realtime-transport.ts:101

Parameters

error

LLMTransportError

Returns

void

Implementation of

LLMTransport.onError

---

onGoAway()?

optional onGoAway: (timeLeft) => void

Defined in: transport/openai-realtime-transport.ts:104

Parameters

timeLeft

string

Returns

void

Implementation of

LLMTransport.onGoAway

---

onGroundingMetadata()?

optional onGroundingMetadata: (metadata) => void

Defined in: transport/openai-realtime-transport.ts:106

Parameters

metadata

Record<string, unknown>

Returns

void

Implementation of

LLMTransport.onGroundingMetadata

---

onInputTranscription()?

optional onInputTranscription: (text) => void

Defined in: transport/openai-realtime-transport.ts:98

Parameters

text

string

Returns

void

Implementation of

LLMTransport.onInputTranscription

---

onInterrupted()?

optional onInterrupted: () => void

Defined in: transport/openai-realtime-transport.ts:97

Returns

void

Implementation of

LLMTransport.onInterrupted

---

onModelTurnStart()?

optional onModelTurnStart: () => void

Defined in: transport/openai-realtime-transport.ts:103

Fires when the model begins any response (audio, tool call, etc.). Used by VoiceSession to trigger STT provider commit.

Returns

void

Implementation of

LLMTransport.onModelTurnStart

---

onOutputTranscription()?

optional onOutputTranscription: (text) => void

Defined in: transport/openai-realtime-transport.ts:99

Parameters

text

string

Returns

void

Implementation of

LLMTransport.onOutputTranscription

---

onRealtimeLLMUsage()?

optional onRealtimeLLMUsage: (usage) => void

Defined in: transport/openai-realtime-transport.ts:110

Optional: fires when the provider reports token or duration usage for billing/observability.

Parameters

usage

RealtimeLLMUsageEvent

Returns

void

Implementation of

LLMTransport.onRealtimeLLMUsage

---

onResumptionUpdate()?

optional onResumptionUpdate: (handle, resumable) => void

Defined in: transport/openai-realtime-transport.ts:105

Parameters

handle

string

resumable

boolean

Returns

void

Implementation of

LLMTransport.onResumptionUpdate

---

onSessionReady()?

optional onSessionReady: (sessionId) => void

Defined in: transport/openai-realtime-transport.ts:100

Parameters

sessionId

string

Returns

void

Implementation of

LLMTransport.onSessionReady

---

onSpeechStarted()?

optional onSpeechStarted: () => void

Defined in: transport/openai-realtime-transport.ts:109

Fires when the transport detects user speech via VAD. Used for TTS-level barge-in when the LLM is idle but TTS is still playing. OpenAI: wired to input_audio_buffer.speech_started. Gemini: may require custom VAD signal — needs empirical testing.

Returns

void

Implementation of

LLMTransport.onSpeechStarted

---

onTextDone()?

optional onTextDone: () => void

Defined in: transport/openai-realtime-transport.ts:108

Fires when the model's text response is complete for this turn. Signals that all text for the current response has been delivered. Ordering contract: fires after all onTextOutput, before onTurnComplete.

Returns

void

Implementation of

LLMTransport.onTextDone

---

onTextOutput()?

optional onTextOutput: (text) => void

Defined in: transport/openai-realtime-transport.ts:107

Fires when the model produces text output (text-mode responses). Only active when responseModality is 'text' (i.e., external TTS in use).

Parameters

text

string

Incremental text chunk (may be partial word/sentence)

Returns

void

Implementation of

LLMTransport.onTextOutput

---

onToolCall()?

optional onToolCall: (calls) => void

Defined in: transport/openai-realtime-transport.ts:94

Parameters

calls

TransportToolCall[]

Returns

void

Implementation of

LLMTransport.onToolCall

---

onToolCallCancel()?

optional onToolCallCancel: (ids) => void

Defined in: transport/openai-realtime-transport.ts:95

Parameters

ids

string[]

Returns

void

Implementation of

LLMTransport.onToolCallCancel

---

onTurnComplete()?

optional onTurnComplete: () => void

Defined in: transport/openai-realtime-transport.ts:96

Returns

void

Implementation of

LLMTransport.onTurnComplete

Accessors

isConnected

Get Signature

get isConnected(): boolean

Defined in: transport/openai-realtime-transport.ts:147

Returns

boolean

Implementation of

LLMTransport.isConnected

Methods

clearAudio()

clearAudio(): void

Defined in: transport/openai-realtime-transport.ts:265

Returns

void

Implementation of

LLMTransport.clearAudio

---

commitAudio()

commitAudio(): void

Defined in: transport/openai-realtime-transport.ts:260

Returns

void

Implementation of

LLMTransport.commitAudio

---

connect()

connect(transportConfig?): Promise<void>

Defined in: transport/openai-realtime-transport.ts:153

Parameters

transportConfig?

LLMTransportConfig

Returns

Promise<void>

Implementation of

LLMTransport.connect

---

disconnect()

disconnect(): Promise<void>

Defined in: transport/openai-realtime-transport.ts:202

Returns

Promise<void>

Implementation of

LLMTransport.disconnect

---

reconnect()

reconnect(state?): Promise<void>

Defined in: transport/openai-realtime-transport.ts:221

Parameters

state?

ReconnectState

Returns

Promise<void>

Implementation of

LLMTransport.reconnect

---

sendAudio()

sendAudio(base64Data): void

Defined in: transport/openai-realtime-transport.ts:255

Parameters

base64Data

string

Returns

void

Implementation of

LLMTransport.sendAudio

---

sendContent()

sendContent(turns, turnComplete?): void

Defined in: transport/openai-realtime-transport.ts:351

Parameters

turns

ContentTurn[]

turnComplete?

boolean = true

Returns

void

Implementation of

LLMTransport.sendContent

---

sendFile()

sendFile(base64Data, mimeType): void

Defined in: transport/openai-realtime-transport.ts:383

Parameters

base64Data

string

mimeType

string

Returns

void

Implementation of

LLMTransport.sendFile

---

sendToolResult()

sendToolResult(result): void

Defined in: transport/openai-realtime-transport.ts:405

Parameters

result

TransportToolResult

Returns

void

Implementation of

LLMTransport.sendToolResult

---

transferSession()

transferSession(config, state?): Promise<void>

Defined in: transport/openai-realtime-transport.ts:302

Parameters

config

SessionUpdate

state?

ReconnectState

Returns

Promise<void>

Implementation of

LLMTransport.transferSession

---

triggerGeneration()

triggerGeneration(instructions?): void

Defined in: transport/openai-realtime-transport.ts:442

Parameters

instructions?

string

Returns

void

Implementation of

LLMTransport.triggerGeneration

---

updateSession()

updateSession(config): void

Defined in: transport/openai-realtime-transport.ts:272

Parameters

config

SessionUpdate

Returns

void

Implementation of

LLMTransport.updateSession