Creating Custom LLM Providers

Add a new LLM provider to Loopstack by implementing LlmProviderInterface and registering it with the LlmProviderRegistry.

Architecture

@loopstack/llm-provider     ← contracts, registry, adapter tools, helpers
    ↑               ↑              ↑
@loopstack/claude  @loopstack/openai  your provider

• @loopstack/llm-provider — shared interfaces, LlmProviderRegistry, adapter tools (LlmGenerateTextTool, LlmGenerateObjectTool, LlmDelegateToolCallsTool, LlmUpdateToolResultTool), shared helpers, and LlmMessageDocument
• Provider modules (e.g. @loopstack/claude, @loopstack/openai) — implement LlmProviderInterface, self-register at module init
• Adapter tools route to the correct provider at runtime based on the provider config value

Implement LlmProviderInterface

import { Injectable, OnModuleInit } from '@nestjs/common';
import type {
  LlmContext,
  LlmGenerateObjectArgs,
  LlmGenerateObjectResult,
  LlmGenerateTextArgs,
  LlmGenerateTextResult,
  LlmNormalizedMessage,
  LlmProviderInterface,
  LlmUsage,
} from '@loopstack/llm-provider';
import { LlmProviderRegistry } from '@loopstack/llm-provider';

@Injectable()
export class OllamaLlmProvider implements LlmProviderInterface, OnModuleInit {
  readonly providerId = 'ollama';

  constructor(private readonly registry: LlmProviderRegistry) {}

  onModuleInit(): void {
    this.registry.register(this);
  }

  async generateText(args: LlmGenerateTextArgs, ctx: LlmContext): Promise<LlmGenerateTextResult> {
    // 1. Resolve messages from ctx.documents (or use args.messages / args.prompt)
    // 2. Call your LLM API
    // 3. Normalize the response to LlmNormalizedMessage format
    // 4. Return { message, response }

    const nativeResponse = await this.callOllamaApi(args, ctx);

    return {
      message: this.normalizeResponse(nativeResponse),
      response: nativeResponse, // preserve native response for round-trips
    };
  }

  async generateObject(args: LlmGenerateObjectArgs, ctx: LlmContext): Promise<LlmGenerateObjectResult> {
    // Similar to generateText, but force structured output
    // Use args.outputSchema to constrain the response

    const nativeResponse = await this.callOllamaStructuredApi(args, ctx);

    return {
      data: nativeResponse.parsedOutput,
      response: nativeResponse,
    };
  }

  extractUsage(response: unknown): LlmUsage | undefined {
    // Extract token usage from the native API response
    const r = response as { usage?: { prompt_tokens: number; completion_tokens: number } };
    if (!r.usage) return undefined;
    return {
      inputTokens: r.usage.prompt_tokens,
      outputTokens: r.usage.completion_tokens,
    };
  }

  toProviderMessage(message: LlmNormalizedMessage): unknown {
    // Convert normalized message back to provider-specific message format
    // Used by resolveMessages() for API round-trips
    return {
      role: message.role,
      content: message.blocks ? message.blocks.map((block) => this.convertBlock(block)) : message.text,
    };
  }
}

The Interface

interface LlmProviderInterface<TProviderConfig = Record<string, unknown>> {
  /** Unique provider identifier (e.g. 'ollama'). Used in config. */
  readonly providerId: string;

  /** Invoke the LLM and return a normalized response. */
  generateText(args: LlmGenerateTextArgs<TProviderConfig>, ctx: LlmContext): Promise<LlmGenerateTextResult>;

  /** Generate a structured object conforming to a JSON Schema. */
  generateObject(args: LlmGenerateObjectArgs<TProviderConfig>, ctx: LlmContext): Promise<LlmGenerateObjectResult>;

  /** Extract usage stats from the native API response. */
  extractUsage(response: unknown): LlmUsage | undefined;

  /** Convert normalized message to provider-specific message format. */
  toProviderMessage(message: LlmNormalizedMessage): unknown;
}

Method responsibilities

Method	Purpose
generateText	Call the LLM API, return normalized LlmNormalizedMessage + native response
generateObject	Same but force structured output matching args.outputSchema
extractUsage	Parse token usage from native response (for logging/quota)
toProviderMessage	Convert a normalized message to provider format (for history this provider did not write)

Replaying history

A conversation may switch provider between turns, so the history your provider reads can hold messages another provider wrote. Follow these rules in your message resolution:

1. Replay meta.response only when you wrote it. LlmGenerateTextTool saves the native response in doc.meta.response and your providerId in doc.meta.provider. Pass the native response back to your API only when doc.meta.provider === this.providerId; any other response is in another provider’s format.
2. Convert everything else with toProviderMessage. Map every block your API can express — text, tool_call, tool_result — and leave out the blocks it cannot, such as another provider’s thinking or server_tool_use / server_tool_result blocks.
3. Reject server tools you cannot run. A server_tool entry in args.tools carries one provider’s native tool spec in config. Throw an error that names the tool instead of dropping it.

const ownResponse = doc.meta?.provider === this.providerId ? doc.meta.response : undefined;
const message = ownResponse
  ? this.fromNativeResponse(ownResponse)
  : this.toProviderMessage(doc.content as LlmNormalizedMessage);

What you DON’T implement

Tool delegation (delegateToolCalls, updateToolResult) is handled by the shared LlmDelegateService and LlmToolsHelperService — they work identically for all providers. You only need to implement the LLM call itself.

LlmContext

The context passed to provider methods:

interface LlmContext {
  /** Runtime documents for the current workflow execution (used for message history). */
  documents: DocumentEntity[];
}

Use ctx.documents with args.messagesSearchTag to resolve message history from saved documents.

LlmGenerateTextArgs

The args your generateText method receives:

Field	Type	Description
system	string?	System prompt
messages	LlmMessage[]?	Explicit messages (alternative to document-based history)
prompt	string?	Simple prompt string
messagesSearchTag	string?	Tag to filter documents as message history
tools	LlmResolvedTool[]?	Tool definitions the LLM can call
model	string?	Model name
providerConfig	TProviderConfig?	Provider-specific config (temperature, maxTokens, etc.)
onStream	LlmStreamHandler?	Optional streaming callback
streamMessageId	string?	Message ID for correlating stream events

Normalized message format

All providers must normalize their responses to LlmNormalizedMessage:

interface LlmNormalizedMessage {
  id?: string;
  role: 'user' | 'assistant';
  text: string;
  blocks?: LlmContentBlock[];
  stopReason?: 'end_turn' | 'tool_use' | 'max_tokens' | 'stop_sequence';
}

text is the plain-text projection (always populated). blocks is the structured form, present when the message contains non-text blocks (thinking, tool calls, tool results).

Content blocks are a union of:

• { type: 'text', text: string } — text output
• { type: 'thinking', text: string } — reasoning/thinking output
• { type: 'tool_call', id: string, name: string, args: Record<string, unknown> } — tool call
• { type: 'tool_result', toolCallId: string, content: string, isError: boolean } — tool result
• { type: 'server_tool_use', id: string, name: string, input: Record<string, unknown> } — a server tool call the provider ran
• { type: 'server_tool_result', toolUseId: string, content: unknown } — the result of that server tool call

Create the module

import { Module } from '@nestjs/common';
import { OllamaLlmProvider } from './ollama-llm-provider';
import { OllamaClientService } from './services/ollama-client.service';

@Module({
  providers: [OllamaClientService, OllamaLlmProvider],
  exports: [OllamaClientService, OllamaLlmProvider],
})
export class OllamaModule {}

Usage

Users import your module — no other changes needed:

@Module({
  imports: [LoopstackModule.forRoot(), OllamaModule],
})
export class AppModule {}

Then use it via config:

const result = await this.llmGenerateText.call(
  { prompt: 'Hello' },
  { config: { provider: 'ollama', model: 'llama3' } },
);

Streaming support

If your provider supports streaming, use the args.onStream callback:

async generateText(args: LlmGenerateTextArgs, ctx: LlmContext): Promise<LlmGenerateTextResult> {
  const stream = this.client.stream(/* ... */);

  if (args.onStream) {
    const messageId = args.streamMessageId ?? crypto.randomUUID();
    await args.onStream({ type: 'start', messageId });

    for await (const chunk of stream) {
      await args.onStream({ type: 'text_delta', messageId, delta: chunk.text });
    }

    const finalMessage = this.normalizeResponse(stream.finalResponse);
    await args.onStream({ type: 'done', messageId, message: finalMessage });
  }

  // Always return the complete final response regardless of streaming
  return { message: finalMessage, response: stream.finalResponse };
}

Key types reference

Type	Description
LlmProviderInterface	Contract for provider implementations
LlmProviderRegistry	Runtime registry — register(), get(), has()
LlmGenerateTextArgs	Input for text generation
LlmGenerateTextResult	Response: { message, response }
LlmGenerateObjectArgs	Input for structured output (includes outputSchema)
LlmGenerateObjectResult	Response: { data, response }
LlmNormalizedMessage	Normalized message: role, content, stopReason
LlmContentBlock	Content block union: text, thinking, tool_call, tool_result, server_tool_use, server_tool_result
LlmStopReason	'end_turn' | 'tool_use' | 'max_tokens' | 'stop_sequence'
LlmToolCall	Normalized tool call: id, name, args
LlmContext	Execution context with documents
LlmUsage	Token usage: inputTokens, outputTokens, optional cache/reasoning
LlmResultMeta	Metadata from adapter tools: provider, model, usage
LlmConfigSchema	Shared Zod schema for model config passthrough
LlmStreamEvent	Stream event union: start, text_delta, thinking_delta, tool_call, done, error
LlmDelegateResult	Tool execution results: allCompleted, toolResults, pendingCount, errorCount, hasErrors, errors