public class Quantizer
Bounded-memory, deterministic weight-only Q8 checkpoint conversion. Floating tensors of rank at least two become physical signed INT8 with separate FP32 scales for groups along the last dimension. Scalars, vectors and non-floating tensors are preserved byte-for-byte, including recurrent control parameters and normalization weights. No model-sized Float copy is made. The output is Klyn Q8 v1, not GGUF Q8_0, AWQ or a drop-in Transformers model.
report = Quantizer.quantize(Path("checkpoint"), Path("checkpoint-q8"))
| Modifier and Type | Member | Description |
|---|---|---|
| public static | quantizequantize(source as Path, destination as Path, options as QuantizationOptions = null) as QuantizationReport |
Converts a local Safetensors file or a Hugging Face checkpoint directory. |