klyn.genai.llm.inference.LocalLLMProvider.prefixCaching
field
public prefixCaching as Boolean = false
Description

Opt-in reuse of the exact initial system/tool-schema frame only. Checkpoints belong to this provider/model/device, never to user turns. Requires additional sequence storage; use clearPrefixCache() to release it. Calls remain serialized on the provider's inference owner.

Example

provider.prefixCaching = true