klyn.genai.llm.inference.LocalLLMProvider.contextCacheMinTokens
field
public contextCacheMinTokens as Long = 128l
Description

Minimum input tokens beyond the static frame before retaining a cache. Avoids a full checkpoint copy for tiny requests; does not truncate input.

Example

provider.contextCacheMinTokens = 128l