public static load(repository as String, quantization as ModelQuantization = ModelQuantization.AUTO, device as Device = Device.auto, maxContext as Long = 2048l, revision as String = "main", offline as Boolean = false, verbose as Boolean = false, token as String = "") as LocalLLMProvider
Downloads the requested Hub variant if necessary, verifies it and loads it on the selected CPU/GPU. No Python, external inference engine or Toolkit is used. AUTO selects an unambiguous executable Q8_0 asset; no silent conversion.
| Parameter | Description |
|---|---|
repository | Hugging Face owner/name. |
quantization | Published GGUF variant; no automatic requantization. |
device | CPU/GPU selector; auto selects a compatible GPU, otherwise CPU. |
maxContext | Prompt plus response bound. |
revision | Hub branch/tag/commit, pinned in the downloaded snapshot. |
offline | Require the complete cache without network access. |
verbose | Show download and weight-loading ProgressBar displays; false by default, producing no console output. |
token | Optional private-repository token, not forwarded across hosts. |
Owned AutoClosable provider; reuse it across requests and sessions.
try provider = LocalLLMProvider.load("bartowski/Qwen_Qwen3.5-9B-GGUF", ModelQuantization.Q8_0): print(LLMChatSession(provider).ask("Hello"))