public override tokenizeLine(line as String, initialState as LexicalState = null) as LineScanResult
Tokenizes a single line of Markdown, tracking multi-line fenced code blocks via LexicalState.
| Parameter | Description |
|---|---|
line | Single line of Markdown text. |
initialState | Lexical state inherited from the previous line. |
LineScanResult with Markdown or delegated code spans and updated LexicalState.
import klyn.code
import klyn.code.markdown
t = MarkdownTokenizer()
s1 = t.tokenizeLine("```klyn", LexicalState.NORMAL)
assert s1.endState.inMarkdownFence()
s2 = t.tokenizeLine("public class Demo:", s1.endState)
assert s2.spans[0].kind == TokenKind.KEYWORD