public override tokenizeLine(line as String, initialState as LexicalState = null) as LineScanResult
Tokenizes a single line of Klyn code with contextual KQL query: state and regex literal support.
| Parameter | Description |
|---|---|
line | Single line of Klyn source text. |
initialState | Lexical state inherited from the previous line. |
LineScanResult with spans and updated LexicalState.
import klyn.code
import klyn.code.klyn
t = KlynTokenizer()
r1 = t.tokenizeLine(" query:", LexicalState.NORMAL)
assert r1.endState.inQueryBlock()
r2 = t.tokenizeLine(" select id from users where active", r1.endState)
assert r2.spans[1].kind == TokenKind.QUERY_KEYWORD