GatedDeltaAttention

classin packageklyn.math.nn.attention

public class GatedDeltaAttention implements AutoClosable

GatedDeltaAttention

All Implemented Interfaces: AutoClosable

Single-token gated delta recurrence with causal depthwise convolution on CPU/GPU. Queries/keys use L2 normalization; recurrent state and accumulations use FP32. Storage is constant with respect to sequence length. This inference state has no training API and never claims to implement an unavailable backward pass. Use a separate instance for each layer/sequence; concurrent use is unsupported.

Example

state = GatedDeltaAttention(16l, 32l, 128l, 128l, 4l)

Constructors

Modifier and Type	Member	Description
public	GatedDeltaAttention `GatedDeltaAttention(keyHeads as Long, valueHeads as Long, keyDim as Long, valueDim as Long, kernelWidth as Long, epsilon as Double = 1e-6)`	Creates an initially zero recurrent state.

Methods

Modifier and Type	Member	Description
public	close `close() as Void`	Releases recurrent device storage; repeated closes are harmless.
public	step `step(output as NDArray<Float>, qkv as NDArray<Float>, gate as NDArray<Float>, alpha as NDArray<Float>, beta as NDArray<Float>, conv as DeviceQuantizedMatrix, bias as NDArray<Float>, decay as NDArray<Float>, norm as NDArray<Float>) as Void`	Updates convolution and recurrence, then applies RMS normalization/SiLU gate.

Inherited Methods

methodInherited Methods from AutoClosable: close