SourceTokenCursor

classin packageklyn.code

public class SourceTokenCursor extends Object

Object

└ SourceTokenCursor

Provides stream navigation and lexical refactoring helpers over tokenized source code.

SourceTokenCursor wraps a Tokenizer to iterate over code tokens while optionally skipping whitespace and comments, extract unique source identifiers for lexical code completion, or safely rename symbols without touching comments or string literals.

import klyn.code
import klyn.code.klyn

cursor = SourceTokenCursor.fromSource(KlynTokenizer(), "x = 1 # x in comment\ny = x + 2")
renamed = SourceTokenCursor.renameIdentifier(KlynTokenizer(), "x = 1 # x\ny = x + 2", "x", "total")
assert "total = 1 # x" in renamed

Inherited Properties

propertyInherited Properties from Object: type

Constructors

Modifier and Type	Member	Description
public	SourceTokenCursor `SourceTokenCursor(tokens as ArrayList<Token>, skipTrivia as Boolean = true)`	Constructs a cursor over a list of `Token`s.

Methods

Modifier and Type	Member	Description
public static	extractIdentifiers `extractIdentifiers(tokenizer as Tokenizer, source as String, minLength as Int = 2) as ArrayList<String>`	Extracts all unique identifiers appearing in executable code (excluding comments and literals).
public static	fromSource `fromSource(tokenizer as Tokenizer, source as String, skipTrivia as Boolean = true) as SourceTokenCursor`	Tokenizes `source` with `tokenizer` and returns a ready-to-use `SourceTokenCursor`.
public	hasNext `hasNext() as Boolean`	Returns `true` when at least one more token is available.
public	next `next() as Token`	Returns the current token and advances the cursor to the next token.
public	peek `peek() as Token`	Returns the current token without advancing the cursor, or `null` at end of stream.
public static	renameIdentifier `renameIdentifier(tokenizer as Tokenizer, source as String, oldName as String, newName as String) as String`	Safely renames all occurrences of an identifier in executable code without modifying comments, documentation blocks, or string/character/regex literals.
public	reset `reset() as Void`	Resets the cursor back to the beginning of the token stream.

Inherited Methods

methodInherited Methods from Object: deepCopy, fromJson, hash, toDict, toJson, toMap, toString, toXml