public GroupedQueryAttention(heads as Long, kvHeads as Long, width as Long, capacity as Long, rotary as Long, epsilon as Double = 1e-6)
Allocates sequence-owned caches. No past tensor is recopied when appending.
| Parameter | Description |
|---|---|
heads | Query head count. |
kvHeads | KV head count dividing heads. |
width | Head dimension. |
capacity | Maximum token count for this sequence. |
rotary | Even rotated width, at most width. |
epsilon | Positive RMS stabilizer. |
state = GroupedQueryAttention(16l, 4l, 256l, 2048l, 64l)