public GpuQuantizedMatrix(reader as GGUFReader, info as GGUFTensor, device as Device, buffer as Byte[], reorder as Int = 0, keyHeads as Long = 1l, valueHeads as Long = 1l, valueDim as Long = 1l)
Loads exact GGUF F32, F16, BF16, Q8_0, Q4_K, Q5_K or Q6_K parameters. Q8 scales widen losslessly to F32; integer codes remain unchanged. Other formats decode only in GPU registers without expansion or requantization. The optional permutation restores logical HF head ordering.
| Parameter | Description |
|---|---|
reader | Validated GGUF descriptor reader. |
info | Selected tensor. |
device | NVIDIA GPU selector. |
buffer | Shared bounded upload buffer. |
reorder | Internal Qwen head-layout permutation selector. |
keyHeads | Linear key head count. |
valueHeads | Linear value head count. |
valueDim | Linear value width. |
matrix = GpuQuantizedMatrix(reader, reader.tensors["output.weight"], Device.gpu[0], buffer)