public tokenizeLine(line as String, initialState as LexicalState = null) as LineScanResult
Tokenizes a single line of source code starting from initialState.
| Parameter | Description |
|---|---|
line | Source line text. |
initialState | Lexical state at the start of line, or null for LexicalState.NORMAL. |
LineScanResult with spans and endState.
import klyn.code
p = LanguageProfile("java", "Java").withLineComments(["//"]).withKeywords(["return"])
t = CodeTokenizer(p)
res = t.tokenizeLine("return 42; // done")
assert res.spans.size > 0