LLM API and Agents

Keep credentials outside source control. The provider URL must be the API version root; the implementation appends /chat/completions automatically.

import klyn.collections
import klyn.llm

def requiredEnvironment(name as String) as String:
    value as String = Application.environmentVariables.get(name, "")
    if value == "":
        throw Exception("Missing environment variable: " + name)
    return value

provider = OpenAIProvider(
    baseUrl=requiredEnvironment("KLYN_LLM_URL"),
    apiKey=requiredEnvironment("KLYN_LLM_API_KEY"),
    model=requiredEnvironment("KLYN_LLM_MODEL"),
    temperature=0.2
)

export KLYN_LLM_URL="https://provider.example/v1"
export KLYN_LLM_API_KEY="..."
export KLYN_LLM_MODEL="model-id"

Use the request objects directly when the caller wants full control over the message list and does not need a managed conversation.

messages as ArrayList<LLMThreadMessage> = [
    LLMThreadMessage.system("Answer in one short paragraph."),
    LLMThreadMessage.user("What is static typing?")
]

response = provider.complete(LLMRequest(messages))
if response.message is not null:
    print(response.message.content)

print("tokens: " + response.totalTokens)

LLMChatSession retains system, user, assistant, and tool messages. Reusing the same session therefore gives the model the preceding conversation without rebuilding history by hand.

session = LLMChatSession(provider) \
    .withSystem("You are a concise Klyn programming assistant.")

print(session.ask("What does f:[1, 2, 3] create?"))
print(session.ask("Is that collection mutable?"))

print("messages: " + session.thread().size)
print("last request tokens: " + session.lastTotalTokens)

Tools are ordinary typed Klyn methods. Declare the externally visible contract on an interface with @Tool, describe each model-provided argument with @Param, then keep the implementation focused on deterministic application logic.

interface InventoryTools:

    @Tool(name="inventory_stock", description="Return the available quantity for one product")
    public stock(
        @Param(name="product", description="Product name")
        product as String
    ) as Int


class InventoryToolsImpl implements InventoryTools:

    private _stock as Map<String, Int> = {
        "keyboard": 12,
        "mouse": 7,
        "monitor": 3
    }

    public override stock(product as String) as Int:
        return this._stock.get(product.toLower(), 0)

Register the implementation, attach it to a session, and ask a goal-oriented question. The session performs the agent loop: it sends the prompt, executes requested tools, appends their results to history, and asks the model again until a final text response is produced.

tools = ToolRegistry.builder() \
    .add(InventoryTools, InventoryToolsImpl()) \
    .build()

agent = LLMChatSession(provider) \
    .withSystem("""You manage a small inventory.
Always call inventory_stock before answering a stock question.
Never invent quantities. Say that an item is unavailable when the tool returns 0.""") \
    .withTools(tools) \
    .withMaxToolCalls(6)

answer = agent.ask("Can I prepare five keyboard and mouse bundles?")
print(answer)

1. The provider receives the tool JSON schema generated from the annotated interface.
2. The model returns one or more ToolCall values.
3. ToolRegistry validates/coerces arguments and invokes the registered Klyn method.
4. The session stores each tool result and sends the augmented history back to the provider.
5. The loop ends when the provider returns an assistant message without tool calls.

Long-running agents should configure the real context size of their model. When token usage reaches the selected percentage, the session can summarize older exchanges while preserving the system message and recent context.

agent.withContextWindow(128000, compactAtPercent=90) \
    .withCompactor(
        "Preserve decisions, tool results, constraints, and unresolved work.",
        threshold=0
    )

if agent.consumeCompactionFlag():
    print("Conversation context was compacted")

Replace 128000 with the context window advertised by the selected model. A LLMToolCallObserver can expose tool progress in a UI, and a LLMCompactionObserver can report compaction before it starts.

• Validate every tool argument again inside the implementation; model output is untrusted input.
• Expose narrow business operations, not a generic shell, unrestricted SQL, or arbitrary file access.
• Use withMaxToolCalls() to bound accidental loops and provider cost.
• Apply authorization in tools. A system prompt is guidance, not an access-control boundary.
• Keep API keys out of source, logs, prompts, tool results, and exception messages.
• Catch provider exceptions at the application boundary and offer cancellation for interactive UIs.

samples/llm.kn demonstrates the low-level request API. samples/Financial.kn is a larger interactive tool-calling agent. Both are listed on the Code Examples page.

Continue with Reflection to inspect types and their members at runtime. KAR and KAB archives close the Advanced Topics section afterward.