Overview > klyn.code.csharp > CSharpTokenizer
Version 0.2.1 Overview Index Tree Deprecated
CSharpTokenizer
classin packageklyn.code.csharp
public class CSharpTokenizer extends CodeTokenizer
Object
└ CodeTokenizer
└ CSharpTokenizer

Specialized CodeTokenizer for C# source files (.cs, .csx).

Supports // and multi-line block comments, #region / #if directives, @"..." verbatim strings, and $"..." interpolated strings.

import klyn.code
import klyn.code.csharp

tokenizer = CSharpTokenizer()
res = tokenizer.tokenizeLine("public async Task<int> LoadAsync() { return 1; }")
assert res.spans[0].kind == TokenKind.KEYWORD
Inherited Properties
propertyInherited Properties from Object: type
Constructors
Modifier and Type Member Description
public CSharpTokenizer
CSharpTokenizer()
 Constructs a CSharpTokenizer configured with CSharpLanguage.profile().
Inherited Methods
methodInherited Methods from CodeTokenizer: appendLiteralSpans, classifyIdentifier, collectIdentifiers, identifierAt, identifierPrefixAt, isAsciiAlpha, isAsciiDigit, isCodePosition, isCodePositionInSource, isEscapedAt, isHexDigit, isIdentPart, isIdentStart, isNumberStart, isOperatorChar, isPreprocessorLine, isPunctuationChar, isValidIdentifier, isValidIdentifier, lineCommentAt, literalDelimiterAt, nextLineState, profile, renameIdentifier, scanEscapeEnd, scanNumberEnd, startsWithAt, startsWithIgnoreCaseAt, stringDelimiterAt, stringPrefixAt, stripLineComments, tokenize, tokenizeLine
methodInherited Methods from Object: deepCopy, fromJson, hash, toDict, toJson, toMap, toString, toXml