Overview > klyn.code.javascript > JsTsTokenizer
Version 0.2.1 Overview Index Tree Deprecated
JsTsTokenizer
classin packageklyn.code.javascript
public class JsTsTokenizer extends CodeTokenizer
Object
└ CodeTokenizer
└ JsTsTokenizer

Specialized CodeTokenizer for JavaScript and TypeScript source files.

Supports JSDoc documentation comments, $ identifiers, template literals, and decorator annotations.

import klyn.code
import klyn.code.javascript

tokenizer = JsTsTokenizer(true)
res = tokenizer.tokenizeLine("export interface User { id: number; }")
assert res.spans[0].kind == TokenKind.KEYWORD
Inherited Properties
propertyInherited Properties from Object: type
Constructors
Modifier and Type Member Description
public JsTsTokenizer
JsTsTokenizer(typescript as Boolean = false)
 Constructs a JsTsTokenizer for JavaScript (typescript = false) or TypeScript (typescript = true).
Inherited Methods
methodInherited Methods from CodeTokenizer: appendLiteralSpans, classifyIdentifier, collectIdentifiers, identifierAt, identifierPrefixAt, isAsciiAlpha, isAsciiDigit, isCodePosition, isCodePositionInSource, isEscapedAt, isHexDigit, isIdentPart, isIdentStart, isNumberStart, isOperatorChar, isPreprocessorLine, isPunctuationChar, isValidIdentifier, isValidIdentifier, lineCommentAt, literalDelimiterAt, nextLineState, profile, renameIdentifier, scanEscapeEnd, scanNumberEnd, startsWithAt, startsWithIgnoreCaseAt, stringDelimiterAt, stringPrefixAt, stripLineComments, tokenize, tokenizeLine
methodInherited Methods from Object: deepCopy, fromJson, hash, toDict, toJson, toMap, toString, toXml