GroupedQueryAttention

classin packageklyn.math.nn.attention

public class GroupedQueryAttention implements AutoClosable

All Implemented Interfaces: AutoClosable

Stateful single-sequence causal grouped-query attention on the selected CPU/GPU. Query projection interleaves a query and sigmoid gate per head. Keys/queries use zero-centered RMS normalization and partial text rotary embeddings. KV caches and linear-size score scratch stay resident on that device. Create one instance per layer and independent sequence; concurrent use is not supported.

Example

attention = GroupedQueryAttention(16l, 4l, 256l, 2048l, 64l)

Properties

Modifier and Type	Member	Description
public readonly property	position `position as Long`	Number of tokens already appended.

Constructors

Modifier and Type	Member	Description
public	GroupedQueryAttention `GroupedQueryAttention(heads as Long, kvHeads as Long, width as Long, capacity as Long, rotary as Long, epsilon as Double = 1e-6)`	Allocates sequence-owned caches.

Methods

Modifier and Type	Member	Description
public	close `close() as Void`	Releases device caches while preserving host values.
public	step `step(output as NDArray<Float>, query as NDArray<Float>, key as NDArray<Float>, value as NDArray<Float>, queryNorm as NDArray<Float>, keyNorm as NDArray<Float>, rotaryTable as NDArray<Float>) as Void`	Appends one projected token and computes attention into a reusable buffer.

Inherited Methods

methodInherited Methods from AutoClosable: close