public generateTokens(notnull prompt as IList<Long>, maxNewTokens as Long = 128l) as ArrayList<Long>
Generates new token IDs by finite-logit argmax, without decoding text. Each invocation creates independent KV and recurrent states. EOS stops generation and is not included in the returned list. No sampling or hidden tokenizer approximation is applied. Prefill is currently sequential.
| Parameter | Description |
|---|---|
prompt | Non-empty token IDs in the model vocabulary. |
maxNewTokens | Non-negative output bound fitting the context. |
Newly generated IDs only, not a copy of the prompt.
tokens = provider.generateTokens([9419l], maxNewTokens=16l)