ModelQuantization

enumin packageklyn.llm.inference

public enum ModelQuantization

ModelQuantization

Hub checkpoint variants. Selecting a variant never silently converts weights or falls back to another precision. Availability and executable formats are distinct: the GGUF CPU/GPU loader supports these variants for Qwen3.5 text models, retaining each tensor's F32/F16/BF16/Q8_0/Q4_K/Q5_K/Q6_K physical encoding. AUTO selects Q8_0 only when its file group is unambiguous.

Example

engine = LocalLLMProvider.load("owner/model-GGUF", ModelQuantization.Q8_0)

Methods

Modifier and Type	Member	Description
public override	toString `toString() as String`	Returns the exact Hub variant name, without the enum type prefix.