bodhi-realtime-agent / OpenAIRealtimeCacheConfig

Interface: OpenAIRealtimeCacheConfig

Defined in: src/transport/openai-realtime-transport.ts:45

OpenAI Realtime cache configuration. Lives on OpenAIRealtimeConfig.cacheConfig.

P3 lands truncation (the documented cache-preservation lever); P5 wires enforcePrefixStability; P6 adds experimental.promptCacheKey.

Extends

• CacheConfigCommon

Properties

allowMutationOnTransfer?

optional allowMutationOnTransfer?: boolean

Defined in: src/types/transport.ts:284

Only consulted when enforcePrefixStability is true. Default: true (multi-agent transfers continue to work). Set false only for hardened single-agent demos that should never legitimately swap instructions.

Inherited from

CacheConfigCommon.allowMutationOnTransfer

---

enforcePrefixStability?

optional enforcePrefixStability?: boolean

Defined in: src/types/transport.ts:277

If true, the OpenAI transport rejects prefix-busting mutations (instructions, tools) that occur AFTER connect() has completed and are NOT part of a transferSession() call. Default: false.

Pre-connect config is always allowed (initial setup never throws). Same-canonical-prefix updates do not throw — other SessionUpdate fields in the same call are still sent on the wire.

Wired in P5 of the configurable context caching design.

Inherited from

CacheConfigCommon.enforcePrefixStability

---

experimental?

optional experimental?: object

Defined in: src/transport/openai-realtime-transport.ts:89

EXPERIMENTAL — sent on every session.update for a given probe scope (baseURL, organization, project, model, promptCacheKey). The first send for a new scope races session.updated (success) against an error event referencing prompt_cache_key or unknown_parameter (rejection). On rejection: probe state for that scope flips to 'rejected', the field is stripped from subsequent sends, and the rejection error is suppressed from the user-facing transport.onError.

Documented for Responses/Chat; community-reported but officially undocumented for Realtime. Combines with the prefix hash for routing affinity (~15 RPM per key per backend host before spillover); over- sharing degrades hit rate. Scope per agent + region.

Not present in the local OpenAI SDK type as of v6.x; emitted via narrow cast in applyOpenAICacheConfig.

promptCacheKey?

optional promptCacheKey?: string

---

truncation?

optional truncation?: "auto" | "disabled" | { retentionRatio: number; tokenLimits?: { postInstructions?: number; }; type: "retention_ratio"; }

Defined in: src/transport/openai-realtime-transport.ts:62

Maps to RealtimeSessionCreateRequest.truncation. Drives whether and how the server truncates conversation history when the model input limit fills. (Limits vary by model — gpt-realtime is 32k input, gpt-realtime-2 is 128k.) Use the retention_ratio object form to preserve more of the cached prefix.

Per the OpenAI SDK documentation: "Truncation will reduce the number of cached tokens on the next turn (busting the cache), since messages are dropped from the beginning of the context. However, clients can also configure truncation to retain messages up to a fraction of the maximum context size, which will reduce the need for future truncations and thus improve the cache rate."

Default: undefined (server default applies — currently 'auto').

Union Members

"auto"

---

"disabled"

---

Type Literal

{ retentionRatio: number; tokenLimits?: { postInstructions?: number; }; type: "retention_ratio"; }

retentionRatio

retentionRatio: number

In [0, 1]. Validated at connect time before opening the WS.

tokenLimits?

optional tokenLimits?: object

Optional fixed token-limit cap.

tokenLimits.postInstructions?

optional postInstructions?: number

type

type: "retention_ratio"