klyn.math.nn.attention.GatedDeltaAttention.step
method
public step(output as NDArray<Float>, qkv as NDArray<Float>, gate as NDArray<Float>, alpha as NDArray<Float>, beta as NDArray<Float>, conv as DeviceQuantizedMatrix, bias as NDArray<Float>, decay as NDArray<Float>, norm as NDArray<Float>) as Void
Description

Updates convolution and recurrence, then applies RMS normalization/SiLU gate.

Parameters
ParameterDescription
outputFloat [1, valueHeads*valueDim] destination.
qkvMixed projection before convolution.
gateSiLU output gate projection.
alphaTime-step projection.
betaUpdate projection.
convPacked causal depthwise weights.
biasTime-step bias.
decayLog decay weights.
normValue-width normalization weights, without zero centering.
Throws
  • ValueException for incompatible buffers or execution scope.
Example

state.step(y, qkv, z, a, b, conv, bias, decay, norm)