public DeviceQuantizedMatrix(reader as GGUFReader, info as GGUFTensor, device as Device, buffer as Byte[], reorder as Int = 0, keyHeads as Long = 1l, valueHeads as Long = 1l, valueDim as Long = 1l)
Loads exact GGUF F32, F16, BF16, Q8_0, Q4_K, Q5_K or Q6_K parameters. Scales are widened losslessly from F16 to F32; integer codes are unchanged. K-quants and 16-bit floats are decoded only in processor registers, not copied into a larger device tensor or requantized into a different variant. Optional head permutation restores the checkpoint's logical HF ordering.
| Parameter | Description |
|---|---|
reader | Validated GGUF descriptor reader. |
info | Selected tensor. |
device | CPU/GPU selector. |
buffer | Shared bounded upload buffer. |
reorder | Internal Qwen head-layout permutation selector. |
keyHeads | Linear key head count. |
valueHeads | Linear value head count. |
valueDim | Linear value width. |
matrix = DeviceQuantizedMatrix(reader, reader.tensors["output.weight"], Device.gpu[0], buffer)