bodhi-realtime-agent / QwenRealtimeTransport
Class: QwenRealtimeTransport
Defined in: src/transport/qwen-realtime-transport.ts:109
Implements
Constructors
Constructor
new QwenRealtimeTransport(
config):QwenRealtimeTransport
Defined in: src/transport/qwen-realtime-transport.ts:156
Parameters
config
Returns
QwenRealtimeTransport
Properties
onAudioOutput?
optionalonAudioOutput?: (base64Data) =>void
Defined in: src/transport/qwen-realtime-transport.ts:138
Parameters
base64Data
string
Returns
void
Implementation of
onClose?
optionalonClose?: (code?,reason?) =>void
Defined in: src/transport/qwen-realtime-transport.ts:147
Parameters
code?
number
reason?
string
Returns
void
Implementation of
onError?
optionalonError?: (error) =>void
Defined in: src/transport/qwen-realtime-transport.ts:146
Parameters
error
error
Error
recoverable
boolean
Returns
void
Implementation of
onFirstAudioChunk?
optionalonFirstAudioChunk?: () =>void
Defined in: src/transport/qwen-realtime-transport.ts:149
Fires once per model response, on the FIRST audio chunk emitted to the client. Distinct from onModelTurnStart (which fires as a response begins, including tool-only turns) — this marks the moment audio actually begins, the anchor for TTS-first-audio / stop-to-first-audio latency. Transports reset their per-response "audio started" flag when a new response begins.
Returns
void
Implementation of
LLMTransport.onFirstAudioChunk
onInputTranscription?
optionalonInputTranscription?: (text) =>void
Defined in: src/transport/qwen-realtime-transport.ts:143
User-speech transcription from the transport's own recognizer.
Parameters
text
string
An incremental delta, not the whole utterance restated. Gemini Live streams many small deltas per utterance (often mid-word); OpenAI and Qwen fire once per utterance from conversation.item.input_audio_transcription.completed, which is the degenerate one-delta case. Consumers must therefore accumulate across calls and reset at each utterance boundary — treating a call as the complete transcript collapses the utterance to its last fragment.
Returns
void
Implementation of
LLMTransport.onInputTranscription
onInterrupted?
optionalonInterrupted?: (serverTurnId?) =>void
Defined in: src/transport/qwen-realtime-transport.ts:142
Parameters
serverTurnId?
number
Monotonic id of the server turn being interrupted (see onTurnComplete). undefined for transports without server turns.
Returns
void
Implementation of
onModelTurnStart?
optionalonModelTurnStart?: () =>void
Defined in: src/transport/qwen-realtime-transport.ts:148
Fires when the model begins a response (audio, text, tool call). Used by VoiceSession to trigger STT provider commit.
Returns
void
Implementation of
onOutputTranscription?
optionalonOutputTranscription?: (text) =>void
Defined in: src/transport/qwen-realtime-transport.ts:144
Parameters
text
string
Returns
void
Implementation of
LLMTransport.onOutputTranscription
onRealtimeLLMUsage?
optionalonRealtimeLLMUsage?: (usage) =>void
Defined in: src/transport/qwen-realtime-transport.ts:154
Optional: fires when the provider reports token or duration usage for billing/observability.
Parameters
usage
RealtimeLLMUsageEvent | null
Returns
void
Implementation of
LLMTransport.onRealtimeLLMUsage
onSessionReady?
optionalonSessionReady?: (sessionId) =>void
Defined in: src/transport/qwen-realtime-transport.ts:145
Parameters
sessionId
string
Returns
void
Implementation of
onSpeechStarted?
optionalonSpeechStarted?: () =>void
Defined in: src/transport/qwen-realtime-transport.ts:152
Fires when the transport detects user speech via VAD. Used for TTS-level barge-in when the LLM is idle but TTS is still playing. OpenAI: wired to input_audio_buffer.speech_started. Gemini: may require custom VAD signal — needs empirical testing.
Returns
void
Implementation of
onTextDone?
optionalonTextDone?: () =>void
Defined in: src/transport/qwen-realtime-transport.ts:151
Fires when the model's text response is complete for this turn. Signals that all text for the current response has been delivered. Ordering contract: fires after all onTextOutput, before onTurnComplete.
Returns
void
Implementation of
onTextOutput?
optionalonTextOutput?: (text) =>void
Defined in: src/transport/qwen-realtime-transport.ts:150
Fires when the model produces text output (text-mode responses). Only active when responseModality is 'text' (i.e., external TTS in use).
Parameters
text
string
Incremental text chunk (may be partial word/sentence)
Returns
void
Implementation of
onToolCall?
optionalonToolCall?: (calls) =>void
Defined in: src/transport/qwen-realtime-transport.ts:139
Parameters
calls
Returns
void
Implementation of
onToolCallCancel?
optionalonToolCallCancel?: (ids) =>void
Defined in: src/transport/qwen-realtime-transport.ts:140
Parameters
ids
string[]
Returns
void
Implementation of
onTurnComplete?
optionalonTurnComplete?: (serverTurnId?) =>void
Defined in: src/transport/qwen-realtime-transport.ts:141
Parameters
serverTurnId?
number
Monotonic id of the server turn that completed, when the transport models server turns explicitly (Gemini external-TTS path). Consumers dedupe finalization by this id. undefined for transports that do not track server turns.
Returns
void
Implementation of
onUserSpeechStopped?
optionalonUserSpeechStopped?: () =>void
Defined in: src/transport/qwen-realtime-transport.ts:153
Fires when the provider's server VAD detects end of user speech. OpenAI/Qwen: wired to input_audio_buffer.speech_stopped. Gemini Live has no equivalent wire event (callback never fires there — the framework falls back to client-VAD timing). The latency anchor derived from this is callback receipt time: biased late by the provider's silence window + network hop (a bounded lower-bound bias — see the observability design, investment-hai-metrics-observability.md §11).
Returns
void
Implementation of
LLMTransport.onUserSpeechStopped
Accessors
audioFormat
Get Signature
get audioFormat():
AudioFormatSpec
Defined in: src/transport/qwen-realtime-transport.ts:202
Returns
Implementation of
capabilities
Get Signature
get capabilities():
TransportCapabilities
Defined in: src/transport/qwen-realtime-transport.ts:198
Static capabilities — read before connecting, used for orchestrator branching.
Returns
Static capabilities — read before connecting, used for orchestrator branching.
Implementation of
isConnected
Get Signature
get isConnected():
boolean
Defined in: src/transport/qwen-realtime-transport.ts:206
Returns
boolean
Implementation of
Methods
cancelResponse()
cancelResponse(
opts?):Promise<void>
Defined in: src/transport/qwen-realtime-transport.ts:361
Cancel the in-flight response — wire-only actuation. May be called from any framework barge-in path that has decided to interrupt. Idempotent.
MUST NOT invoke onInterrupted or any other framework callback — the caller is responsible for finalizeTurn(interrupted). The implementation only sends wire messages and updates the transport's own state.
MUST NOT reject. Transient send failures are caught and logged internally; the returned promise still resolves so fire-and-forget callers cannot trigger unhandled-rejection warnings.
When truncate.audioEndMs is supplied and the transport tracks an active assistant audio item, it also sends a per-provider truncate (OpenAI: conversation.item.truncate). truncate: 'generated' asks the transport to compute audioEndMs from its own per-response generated-audio counter (e.g. OpenAI's audioOutputMs); transports without such a counter ignore the sentinel. Omitting truncate only skips truncation — cancelResponse still stops generation if a response is in flight.
When waitForDone is true, the returned promise resolves only after the trailing response.done(status:'cancelled') arrives from the provider (or a 2000 ms timeout, whichever comes first). Use this before sending a new response.create so cancel and create cannot race. Default false.
Parameters
opts?
CancelResponseOptions
Returns
Promise<void>
Implementation of
clearAudio()
clearAudio():
void
Defined in: src/transport/qwen-realtime-transport.ts:351
Returns
void
Implementation of
clearInputAudio()
clearInputAudio():
void
Defined in: src/transport/qwen-realtime-transport.ts:355
Clear the provider's pending input-audio buffer (OpenAI Realtime: input_audio_buffer.clear). Optional — Gemini and other transports without a server-side append-then-commit input buffer omit this. Called by VoiceSession at grace-window arming to discard any pre-arming echo residue; safe to call when the buffer is empty.
Returns
void
Implementation of
commitAudio()
commitAudio():
void
Defined in: src/transport/qwen-realtime-transport.ts:346
Returns
void
Implementation of
connect()
connect(
transportConfig?):Promise<void>
Defined in: src/transport/qwen-realtime-transport.ts:212
Parameters
transportConfig?
Returns
Promise<void>
Implementation of
disconnect()
disconnect():
Promise<void>
Defined in: src/transport/qwen-realtime-transport.ts:299
Returns
Promise<void>
Implementation of
reconnect()
reconnect(
state?):Promise<void>
Defined in: src/transport/qwen-realtime-transport.ts:322
Parameters
state?
Returns
Promise<void>
Implementation of
replayHistory()
replayHistory(
items):void
Defined in: src/transport/qwen-realtime-transport.ts:663
Replay a conversation into the live session: the host's initial history, and the conversation of a replacement connection.
Parameters
items
readonly ReplayItem[]
Returns
void
Implementation of
sendAudio()
sendAudio(
base64Data):void
Defined in: src/transport/qwen-realtime-transport.ts:342
Parameters
base64Data
string
Returns
void
Implementation of
sendContent()
sendContent(
turns,turnComplete?):void
Defined in: src/transport/qwen-realtime-transport.ts:437
Parameters
turns
turnComplete?
boolean
Returns
void
Implementation of
sendFile()
sendFile(
_base64Data,_mimeType):void
Defined in: src/transport/qwen-realtime-transport.ts:452
Parameters
_base64Data
string
_mimeType
string
Returns
void
Implementation of
sendToolResult()
sendToolResult(
result):void
Defined in: src/transport/qwen-realtime-transport.ts:458
Parameters
result
Returns
void
Implementation of
transferSession()
transferSession(
config,_state?):Promise<void>
Defined in: src/transport/qwen-realtime-transport.ts:408
Parameters
config
_state?
Returns
Promise<void>
Implementation of
triggerGeneration()
triggerGeneration(
instructions?):void
Defined in: src/transport/qwen-realtime-transport.ts:524
Trigger a model response.
Parameters
instructions?
string
Optional one-off instruction (passed as response.create.response.instructions on OpenAI — does not mutate the session prefix, so it is cache-safe for subsequent turns).
Returns
void
Implementation of
LLMTransport.triggerGeneration
updateSession()
updateSession(
config):Promise<void>
Defined in: src/transport/qwen-realtime-transport.ts:382
Apply a session update. Pre-connect: state-only mutation; coalesces with prior pre-connect calls and resolves immediately. The merged config is sent in the single session.update issued at connect time. Post-connect: serialized via the transport's internal FIFO queue; each call produces one wire session.update and awaits its ack. A custom transport may instead apply the update synchronously and return void.
Parameters
config
Returns
Promise<void>