public override tokenizeLine(line as String, initialState as LexicalState = null) as LineScanResult
Tokenizes a single line of HTML/XML markup.
| Parameter | Description |
|---|---|
line | Source line text. |
initialState | Lexical state inherited from the previous line. |
LineScanResult with markup spans and updated LexicalState.
import klyn.code
import klyn.code.markup
t = MarkupTokenizer()
res = t.tokenizeLine("<a href=\"https://klyn.dev\">Klyn</a>")
assert res.spans[1].kind == TokenKind.TAG