bodhi-realtime-agent / GeminiLiveTransport
Class: GeminiLiveTransport
Defined in: src/transport/gemini-live-transport.ts:614
WebSocket transport layer for the Gemini Live API.
Wraps the @google/genai SDK's live.connect() to manage the bidirectional audio stream. Handles connection setup, message routing, tool declaration conversion (Zod → JSON Schema), and session resumption.
Implements LLMTransport for provider-agnostic usage. The constructor callback pattern is preserved for backward compatibility alongside the LLMTransport callback properties.
Implements
Constructors
Constructor
new GeminiLiveTransport(
config,callbacks):GeminiLiveTransport
Defined in: src/transport/gemini-live-transport.ts:824
Parameters
config
GeminiTransportConfig | GeminiProviderTransportConfig
callbacks
Returns
GeminiLiveTransport
Properties
audioFormat
readonlyaudioFormat:AudioFormatSpec
Defined in: src/transport/gemini-live-transport.ts:780
Implementation of
capabilities
readonlycapabilities:TransportCapabilities
Defined in: src/transport/gemini-live-transport.ts:745
Static capabilities — read before connecting, used for orchestrator branching.
Implementation of
onAudioOutput?
optionalonAudioOutput?: (base64Data) =>void
Defined in: src/transport/gemini-live-transport.ts:790
Parameters
base64Data
string
Returns
void
Implementation of
onClose?
optionalonClose?: (code?,reason?) =>void
Defined in: src/transport/gemini-live-transport.ts:799
Parameters
code?
number
reason?
string
Returns
void
Implementation of
onConnectionLifecycle?
optionalonConnectionLifecycle?: (event) =>void=undefined
Defined in: src/transport/gemini-live-transport.ts:820
Connection-lifecycle facts, property form (VoiceSession wires this one).
Parameters
event
Returns
void
Implementation of
LLMTransport.onConnectionLifecycle
onCredentialExpiry?
optionalonCredentialExpiry?: () =>void
Defined in: src/transport/gemini-live-transport.ts:822
The current connection's credential expired (credentialProvider only).
Returns
void
Implementation of
LLMTransport.onCredentialExpiry
onError?
optionalonError?: (error) =>void
Defined in: src/transport/gemini-live-transport.ts:798
Parameters
error
Returns
void
Implementation of
onFirstAudioChunk?
optionalonFirstAudioChunk?: () =>void
Defined in: src/transport/gemini-live-transport.ts:804
Fires once per model response, on the FIRST audio chunk emitted to the client. Distinct from onModelTurnStart (which fires as a response begins, including tool-only turns) — this marks the moment audio actually begins, the anchor for TTS-first-audio / stop-to-first-audio latency. Transports reset their per-response "audio started" flag when a new response begins.
Returns
void
Implementation of
LLMTransport.onFirstAudioChunk
onGenerationComplete?
optionalonGenerationComplete?: () =>void
Defined in: src/transport/gemini-live-transport.ts:803
Returns
void
onGenerationEnd?
optionalonGenerationEnd?: (generationId,reason) =>void
Defined in: src/transport/gemini-live-transport.ts:802
That generation closed, and why. Fires exactly once per start.
Parameters
generationId
string
reason
GenerationEndReason
Returns
void
Implementation of
onGenerationStart?
optionalonGenerationStart?: (generationId) =>void
Defined in: src/transport/gemini-live-transport.ts:801
A model generation opened. Paired with exactly one onGenerationEnd. A generation is one answer: it outlives the provider's turn boundary, because a tool call that finishes the answer can arrive after it.
Parameters
generationId
string
Returns
void
Implementation of
LLMTransport.onGenerationStart
onGoAway?
optionalonGoAway?: (timeLeft) =>void
Defined in: src/transport/gemini-live-transport.ts:805
Parameters
timeLeft
string
Returns
void
Implementation of
onGroundingMetadata?
optionalonGroundingMetadata?: (metadata) =>void
Defined in: src/transport/gemini-live-transport.ts:807
Parameters
metadata
Record<string, unknown>
Returns
void
Implementation of
LLMTransport.onGroundingMetadata
onInputTranscription?
optionalonInputTranscription?: (text) =>void
Defined in: src/transport/gemini-live-transport.ts:795
User-speech transcription from the transport's own recognizer.
Parameters
text
string
An incremental delta, not the whole utterance restated. Gemini Live streams many small deltas per utterance (often mid-word); OpenAI and Qwen fire once per utterance from conversation.item.input_audio_transcription.completed, which is the degenerate one-delta case. Consumers must therefore accumulate across calls and reset at each utterance boundary — treating a call as the complete transcript collapses the utterance to its last fragment.
Returns
void
Implementation of
LLMTransport.onInputTranscription
onInterrupted?
optionalonInterrupted?: (serverTurnId?) =>void
Defined in: src/transport/gemini-live-transport.ts:794
Parameters
serverTurnId?
number
Monotonic id of the server turn being interrupted (see onTurnComplete). undefined for transports without server turns.
Returns
void
Implementation of
onModelTurnStart?
optionalonModelTurnStart?: (generationId?) =>void
Defined in: src/transport/gemini-live-transport.ts:800
Fires when the model begins a response (audio, text, tool call). Used by VoiceSession to trigger STT provider commit.
Parameters
generationId?
string
The generation that opened, on transports that model generations (GeminiLiveTransport: once per generation, with the same id as onGenerationStart, so a tool call after turnComplete that finishes the same answer does not fire it again); undefined elsewhere.
Returns
void
Implementation of
onOutputTranscription?
optionalonOutputTranscription?: (text) =>void
Defined in: src/transport/gemini-live-transport.ts:796
Parameters
text
string
Returns
void
Implementation of
LLMTransport.onOutputTranscription
onRealtimeLLMUsage?
optionalonRealtimeLLMUsage?: (usage) =>void
Defined in: src/transport/gemini-live-transport.ts:811
Optional: fires when the provider reports token or duration usage for billing/observability.
Parameters
usage
Returns
void
Implementation of
LLMTransport.onRealtimeLLMUsage
onResumptionUpdate?
optionalonResumptionUpdate?: (handle,resumable) =>void
Defined in: src/transport/gemini-live-transport.ts:806
Parameters
handle
string
resumable
boolean
Returns
void
Implementation of
LLMTransport.onResumptionUpdate
onSessionReady?
optionalonSessionReady?: (sessionId) =>void
Defined in: src/transport/gemini-live-transport.ts:797
Parameters
sessionId
string
Returns
void
Implementation of
onSpeechStarted?
optionalonSpeechStarted?: () =>void
Defined in: src/transport/gemini-live-transport.ts:810
Fires when the transport detects user speech via VAD. Used for TTS-level barge-in when the LLM is idle but TTS is still playing. OpenAI: wired to input_audio_buffer.speech_started. Gemini: may require custom VAD signal — needs empirical testing.
Returns
void
Implementation of
onTextDone?
optionalonTextDone?: () =>void
Defined in: src/transport/gemini-live-transport.ts:809
Fires when the model's text response is complete for this turn. Signals that all text for the current response has been delivered. Ordering contract: fires after all onTextOutput, before onTurnComplete.
Returns
void
Implementation of
onTextOutput?
optionalonTextOutput?: (text) =>void
Defined in: src/transport/gemini-live-transport.ts:808
Fires when the model produces text output (text-mode responses). Only active when responseModality is 'text' (i.e., external TTS in use).
Parameters
text
string
Incremental text chunk (may be partial word/sentence)
Returns
void
Implementation of
onToolCall?
optionalonToolCall?: (calls) =>void
Defined in: src/transport/gemini-live-transport.ts:791
Parameters
calls
Returns
void
Implementation of
onToolCallCancel?
optionalonToolCallCancel?: (ids) =>void
Defined in: src/transport/gemini-live-transport.ts:792
Parameters
ids
string[]
Returns
void
Implementation of
onTurnComplete?
optionalonTurnComplete?: (serverTurnId?) =>void
Defined in: src/transport/gemini-live-transport.ts:793
Parameters
serverTurnId?
number
Monotonic id of the server turn that completed, when the transport models server turns explicitly (Gemini external-TTS path). Consumers dedupe finalization by this id. undefined for transports that do not track server turns.
Returns
void
Implementation of
onUsageMetadata?
optionalonUsageMetadata?: (usage) =>void=undefined
Defined in: src/transport/gemini-live-transport.ts:818
Raw Live usage, property form (VoiceSession wires this one). Typed to the neutral shape so it satisfies LLMTransport; the object passed is the full provider payload, castable to LiveUsageMetadata.
Parameters
usage
Returns
void
Implementation of
Accessors
currentDialGen
Get Signature
get currentDialGen():
number
Defined in: src/transport/gemini-live-transport.ts:1398
Dial counter: advances per dial and when the current dial fails.
Returns
number
Dial counter: advances per dial and when the current dial fails.
Implementation of
currentTransportGeneration
Get Signature
get currentTransportGeneration():
number
Defined in: src/transport/gemini-live-transport.ts:1402
Post-setup generation counter (advances only on a completed setup).
Returns
number
Post-setup generation counter (advances only on a completed setup).
Implementation of
LLMTransport.currentTransportGeneration
isConnected
Get Signature
get isConnected():
boolean
Defined in: src/transport/gemini-live-transport.ts:1631
Returns
boolean
Implementation of
Methods
abortIncumbent()
abortIncumbent():
Promise<"closed"|"forced">
Defined in: src/transport/gemini-live-transport.ts:1418
Synchronously strand the incumbent connection: advance the dial generation (its callbacks go stale, an in-flight reconnect() continuation will not dial, and a later-resolving dial closes its own session), detach the session before any await, reset server-turn state and discard both resumption handle copies — a redial must never resume the abandoned server-side session. Returns a close bounded by 5 s: the detached session's close() or, when no session is attached, the close a disconnect() already has in flight (that session was detached before this call, and disconnect() alone would await it without a bound). Resolves 'closed' when that close completed in time (at once when nothing is closing), 'forced' on timeout or close error; never rejects. Does not go through disconnect().
Returns
Promise<"closed" | "forced">
Implementation of
cancelResponse()
cancelResponse():
Promise<void>
Defined in: src/transport/gemini-live-transport.ts:1893
Gemini has no cancel-generation wire command — its interrupts are provider-driven via serverContent.interrupted and it keeps streaming the rest of the already-generated response. So cancelResponse() satisfies the widened contract ("stop the current response reaching the user") by suppressing the current server turn's remaining outbound audio until the next server turn (the response to the barge-in) begins. Drives the bufferedUncancellableAudio capability.
Returns
Promise<void>
Implementation of
clearAudio()
clearAudio():
void
Defined in: src/transport/gemini-live-transport.ts:1884
No-op for V1 — server VAD only.
Returns
void
Implementation of
clearResumption()
clearResumption():
void
Defined in: src/transport/gemini-live-transport.ts:1452
Drop both resumption handle copies: the effective handle the next dial resumes with and the legacy config.resumptionHandle alias. The next dial then opens a fresh server session. The connection is untouched.
Returns
void
Implementation of
commitAudio()
commitAudio():
void
Defined in: src/transport/gemini-live-transport.ts:1881
No-op for V1 — server VAD only.
Returns
void
Implementation of
connect()
connect(
transportConfig?):Promise<void>
Defined in: src/transport/gemini-live-transport.ts:881
Establish a WebSocket connection to the Gemini Live API. Resolves only after Gemini sends setupComplete, so callers can safely send content immediately after awaiting this method.
Also satisfies LLMTransport.connect(config) — if config is provided, it is applied before connecting.
Parameters
transportConfig?
Returns
Promise<void>
Implementation of
disconnect()
disconnect():
Promise<void>
Defined in: src/transport/gemini-live-transport.ts:1457
Returns
Promise<void>
Implementation of
getActiveServerTurnId()
getActiveServerTurnId():
number|undefined
Defined in: src/transport/gemini-live-transport.ts:2082
The id of the server turn currently being generated, or undefined when no server turn is active. Active-only by contract: between turns (idle / closed) _serverTurnId still holds the previous turn's value, so it must not be exposed — a stale id would mis-bind a freshly-born Turn.
Returns
number | undefined
Implementation of
LLMTransport.getActiveServerTurnId
getDiagnostics()
getDiagnostics():
TransportDiagnostics
Defined in: src/transport/gemini-live-transport.ts:1636
Snapshot, not a live reference — safe for a caller to hold across ticks.
Returns
Implementation of
getLastNonResumableAt()
getLastNonResumableAt():
number|null
Defined in: src/transport/gemini-live-transport.ts:739
Telemetry helper: the wall-clock ms of the most recent resumable: false sessionResumptionUpdate observed, or null if none has fired.
Returns
number | null
quiesce()
quiesce():
Promise<void>
Defined in: src/transport/gemini-live-transport.ts:772
Pause the transport without disconnecting:
- Cancel any in-flight model response (provider chooses how).
- Suppress onAudioOutput deltas until unquiesce() is called.
- Leave the WebSocket open and conversation state intact.
Idempotent. Used by VoiceSession to transition into transcription mode without tearing the transport down. When omitted, VoiceSession falls back to its framework-layer audio-output guard.
Returns
Promise<void>
Implementation of
reconnect()
reconnect(
stateOrHandle?):Promise<void>
Defined in: src/transport/gemini-live-transport.ts:1343
Disconnect and reconnect, optionally with a new resumption handle or ReconnectState. Accepts either a string handle (legacy API) or ReconnectState (LLMTransport API).
Parameters
stateOrHandle?
string | ReconnectState
Returns
Promise<void>
Implementation of
replayHistory()
replayHistory(
items):void
Defined in: src/transport/gemini-live-transport.ts:2011
Replay a conversation into the live session as one quiet client-content batch (no response requested): the host's initial history, and the conversation of a replacement connection.
Parameters
items
readonly ReplayItem[]
Returns
void
Implementation of
replayUserTurn()
replayUserTurn(
turn):boolean
Defined in: src/transport/gemini-live-transport.ts:1845
Replay a retained user utterance as a complete user turn — inline audio content with an explicit turnComplete, deliberately NOT the realtime channel (no server-VAD dependence; a brief utterance after a barge-in is exactly what the server VAD dropped). Phase 0 validated this shape in both clean and post-barge-in states (raw PCM, no input transcription emitted — see design-retained-user-content-recovery.md).
Parameters
turn
RetainedUserTurn
Returns
boolean
Implementation of
sendAudio()
sendAudio(
base64Data):void
Defined in: src/transport/gemini-live-transport.ts:1567
Send base64-encoded PCM audio to Gemini as realtime input.
Parameters
base64Data
string
Returns
void
Implementation of
sendClientContent()
sendClientContent(
turns,turnComplete?):void
Defined in: src/transport/gemini-live-transport.ts:1604
Send text-based conversation turns to Gemini.
Parameters
turns
object[]
turnComplete?
boolean = true
Returns
void
Deprecated
Prefer sendContent() for framework text turns. Live generation text should use realtime input; keep this only for legacy callers and context replay/prefill flows that need client-content ordering semantics.
sendContent()
sendContent(
turns,turnComplete?):void
Defined in: src/transport/gemini-live-transport.ts:1655
Send provider-neutral content turns to Gemini. Converts ContentTurn to Gemini format.
Parameters
turns
turnComplete?
boolean = true
Returns
void
Implementation of
sendFile()
sendFile(
base64Data,mimeType):void
Defined in: src/transport/gemini-live-transport.ts:1737
Send an image or audio file as realtime input.
Gemini Live's realtime input has separate slots per media kind, not a generic one (the SDK's media field maps to the deprecated media_chunks wire format, which Gemini 3.1 rejects with close code 1007):
image/* → video (an image is a single-frame video) audio/* → audio (the slot sendAudio uses) other → a warning, nothing sent, counted as unsupportedMime
Use sendInlineFile for other file types (documents), which go inline in client content instead.
Parameters
base64Data
string
mimeType
string
Returns
void
Implementation of
sendInlineFile()
sendInlineFile(
base64Data,mimeType):void
Defined in: src/transport/gemini-live-transport.ts:1772
Send a file as inline data in client content (turnComplete: false), for any MIME type, documents included.
Counted by media kind: audio/* on the audio slot, everything else on the video slot. Every MIME type is sent inline here, so none counts as unsupportedMime. A client upload frame is unchecked input, so a missing mimeType counts on the video slot and the data is sent as received.
Parameters
base64Data
string
mimeType
string
Returns
void
Implementation of
sendLiveText()
sendLiveText(
turns):boolean
Defined in: src/transport/gemini-live-transport.ts:1690
Send live text as realtime input (sendRealtimeInput({ text })) on every model, unlike sendContent, whose realtime path depends on the model name. Several turns are joined with newlines, each prefixed with its role (assistant becomes model) so the model can tell them apart; a single turn is sent as is. Turns with no text are skipped, and there is no turnComplete: server activity detection decides the turn boundary.
Returns true when the send was dispatched or accepted into the wind-down buffer (flushed best effort, like the other buffered sends), and false when there is no text, no session, or the send threw. Never throws.
Parameters
turns
Returns
boolean
Implementation of
sendToolResponse()
sendToolResponse(
responses,scheduling?):void
Defined in: src/transport/gemini-live-transport.ts:1584
Send tool execution results back to Gemini (legacy API).
Parameters
responses
object[]
scheduling?
"WHEN_IDLE" | "INTERRUPT" | "SILENT"
Returns
void
sendToolResult()
sendToolResult(
result):void
Defined in: src/transport/gemini-live-transport.ts:1795
Send a tool result back to Gemini (LLMTransport API).
Parameters
result
Returns
void
Implementation of
transferSession()
transferSession(
config,state?):Promise<void>
Defined in: src/transport/gemini-live-transport.ts:1942
Transfer session: update config → reconnect → replay conversation history.
Parameters
config
state?
Returns
Promise<void>
Implementation of
triggerGeneration()
triggerGeneration(
_instructions?):void
Defined in: src/transport/gemini-live-transport.ts:1818
No-op for Gemini — generation is automatic after tool results and content injection.
Parameters
_instructions?
string
Returns
void
Implementation of
LLMTransport.triggerGeneration
unquiesce()
unquiesce():
Promise<void>
Defined in: src/transport/gemini-live-transport.ts:776
Resume normal operation after quiesce(). After this resolves, onAudioOutput fires again on the next response. Idempotent.
Returns
Promise<void>
Implementation of
updateGoogleSearch()
updateGoogleSearch(
enabled):void
Defined in: src/transport/gemini-live-transport.ts:1627
Update Google Search grounding flag (applied on next reconnect).
Parameters
enabled
boolean
Returns
void
updateSession()
updateSession(
config):Promise<void>
Defined in: src/transport/gemini-live-transport.ts:1915
Update session configuration (applied on next reconnect for Gemini — no in-place mutation, capabilities.inPlaceSessionUpdate is false). Async signature for LLMTransport interface parity; body is synchronous.
Parameters
config
Returns
Promise<void>
Implementation of
updateSystemInstruction()
updateSystemInstruction(
instruction):void
Defined in: src/transport/gemini-live-transport.ts:1622
Update the system instruction (applied on next reconnect).
Parameters
instruction
string
Returns
void
updateTools()
updateTools(
tools):void
Defined in: src/transport/gemini-live-transport.ts:1617
Update the tool declarations (applied on next reconnect).
Parameters
tools
Returns
void