Text Search

Collecting Results

Path, FilePath, and FolderPath share the same engine. Files.searchText(path, ...) delegates to that API.

import klyn.io

options = SearchOptions()
options.includes.add("*.kn")
options.smartCase = true
options.maxTotalMatches = 100ul

for hit in Path.cwd.searchText("TODO", options):
    print(f"{hit.path}:{hit.line}:{hit.column}: {hit.text}")

Each SearchMatch contains path, line, column, endColumn, text, and value. Coordinates are one-based Unicode scalar positions. endColumn is exclusive; the line text excludes CR/LF terminators. Multiple non-overlapping occurrences on the same line produce separate results.

Streaming and Cancellation

import klyn.io

options = SearchOptions()
options.workers = 4u

def consume(hit as SearchMatch) as Boolean:
    print(f"{hit.path}:{hit.line}: {hit.value}")
    return false

count = Path.cwd.searchText("TODO", options, consume)
assert count <= 1ul

Callbacks are serialized, even with several workers. Returning false stops delivery globally; the stopping match is included in the returned count. maxTotalMatches applies across all files. Workers stop cooperatively between decoded blocks and deliveries. With multiple workers, callbacks execute on workers and file order is unspecified. One worker uses depth-first traversal, in filesystem enumeration order.

The raw buffer is 64 KiB. Decoding retains incomplete characters across blocks, and the line buffer grows geometrically only when required. Memory depends on the traversal frontier and the longest line, not on the entire contents of all files. Only matched lines become String instances. The collecting overload additionally retains its results; use streaming for large searches.

Text Encodings

The default encoding = "auto" recognizes UTF-8, UTF-16LE/BE and UTF-32LE/BE BOMs, strips the BOM, then decodes incrementally. Without a BOM, UTF-8 is selected. Detection is deliberately not a guess: many byte sequences are valid in several encodings.

import klyn.io

options = SearchOptions()
options.encoding = "windows-1252"
hits = FilePath("legacy.log").searchText("error", options)

try reader = FileReader("unicode.txt", encoding="auto"):
    print(reader.readLine())

Use an explicit charset for BOM-less UTF-16/32 or legacy text. A specified encoding that contradicts a BOM is rejected. UTF-16/32 without an explicit byte order require a BOM. Malformed Unicode and incomplete characters at EOF raise IOException.

Legacy charset names use the system converter: iconv on Unix-like systems, Windows code pages on Windows. Common portable choices include iso-8859-1, windows-1252, and shift-jis. Other charset availability is platform-dependent; unsupported encodings raise IOException rather than silently producing incorrect text. Stateful legacy encodings are not currently supported by the Windows backend.

Binary detection examines NUL bytes in the initial 512-byte sample, after encoding detection; UTF-16/32 zero bytes therefore do not cause false binary classification. Set skipBinary = false to search text containing U+0000. This initial sample is a heuristic, not a complete binary-file validator.

Filters and Regular Expressions

import klyn.io

options = SearchOptions()
options.regex = true
options.includes.add("**/*.log")
options.excludes.add("vendor/")
hits = FolderPath("logs").searchText(r"ERROR\s+\d+", options)

Patterns are literal by default. Nonempty, case-sensitive literals use a shared KMP prefix table for linear-time matching without per-line UTF-8 conversion. regex = true uses the existing klyn.regex.RegEx engine. caseSensitive defaults to true; smartCase selects sensitivity from uppercase characters in the pattern. Case-insensitive matching follows RegEx option bit 1.

Include and exclude globs are relative to the search root. A slashless glob matches a basename; * and ? do not cross separators, while ** does. Directory patterns end in /. Exclusions are tested before descending into directories; includes select files without pruning their parents.

Nested .gitignore files are honored by default, in rule order, with comments, escapes and ! negation. An ignored directory is not traversed: a rule inside that directory cannot re-include it. .git, .cache, node_modules, and build are default exclusions; clear options.excludes to remove them. Dot-prefixed entries require includeHidden = true. Symlinks are never followed. Read failures and callback exceptions cancel the search and propagate to the caller.