Overview > klyn.code.sql > SqlTokenizer
Version 0.2.1 Overview Index Tree Deprecated
SqlTokenizer
classin packageklyn.code.sql
public class SqlTokenizer extends CodeTokenizer
Object
└ CodeTokenizer
└ SqlTokenizer

Specialized CodeTokenizer for SQL scripts and queries (.sql, .ddl, .dml, .psql).

Supports ANSI/PostgreSQL/MySQL/SQLite/T-SQL case-insensitive keywords, -- and # line comments, multi-line block comments, '...' string literals, and "..." / ... quoted identifiers.

import klyn.code
import klyn.code.sql

tokenizer = SqlTokenizer()
res = tokenizer.tokenizeLine("SELECT u.id, COUNT(*) FROM users u WHERE u.active = TRUE;")
assert res.spans[0].kind == TokenKind.KEYWORD
Inherited Properties
propertyInherited Properties from Object: type
Constructors
Modifier and Type Member Description
public SqlTokenizer
SqlTokenizer()
 Constructs a SqlTokenizer configured with SqlLanguage.profile().
Inherited Methods
methodInherited Methods from CodeTokenizer: appendLiteralSpans, classifyIdentifier, collectIdentifiers, identifierAt, identifierPrefixAt, isAsciiAlpha, isAsciiDigit, isCodePosition, isCodePositionInSource, isEscapedAt, isHexDigit, isIdentPart, isIdentStart, isNumberStart, isOperatorChar, isPreprocessorLine, isPunctuationChar, isValidIdentifier, isValidIdentifier, lineCommentAt, literalDelimiterAt, nextLineState, profile, renameIdentifier, scanEscapeEnd, scanNumberEnd, startsWithAt, startsWithIgnoreCaseAt, stringDelimiterAt, stringPrefixAt, stripLineComments, tokenize, tokenizeLine
methodInherited Methods from Object: deepCopy, fromJson, hash, toDict, toJson, toMap, toString, toXml