klyn.math.nn.attention.GroupedQueryAttention.step
method
public step(output as NDArray<Float>, query as NDArray<Float>, key as NDArray<Float>, value as NDArray<Float>, queryNorm as NDArray<Float>, keyNorm as NDArray<Float>, rotaryTable as NDArray<Float>) as Void
Description

Appends one projected token and computes attention into a reusable buffer.

Parameters
ParameterDescription
outputFloat [1, heads*width] result.
queryInterleaved [1, heads*2*width] projection.
key[1, kvHeads*width] key projection.
valueMatching value projection.
queryNormZero-centered head-width weight vector.
keyNormZero-centered head-width weight vector.
rotaryTablePrecomputed [capacity, rotary] cos/sin table.
Throws
  • ValueException when capacity or device/buffer contracts fail.
Example

state.step(output, q, k, v, qNorm, kNorm, rotaryTable)