llm skill

Overview

This skill provides a unified interface to multiple large language model providers including OpenAI, Anthropic, Google, and local models. It simplifies switching between providers, comparing outputs, and managing model-specific details like token usage and costs. The integration focuses on consistent chat behavior and streaming responses for low-latency applications.

How this skill works

The skill routes prompts to the selected provider using provider-specific adapters while exposing a consistent chat API. It can query multiple models in parallel to produce model comparisons, measure token usage, and estimate cost based on provider pricing. Streaming support forwards incremental model outputs to your client, and local adapters let you run inference on on-prem models like Ollama or LM Studio.

When to use it

• You need to compare outputs from different LLM providers for accuracy or style.
• You want a single API to switch between cloud and local models without rewriting logic.
• You must estimate token counts or costs before sending large prompts.
• You need streaming responses for chat UIs or real-time apps.
• You want to run experiments across OpenAI, Anthropic, Google, and local models.

Best practices

• Normalize prompts and system messages to reduce variability when comparing models.
• Run small calibration tests to measure tokens and latency per model before production.
• Prefer streaming for user-facing chat to improve perceived responsiveness.
• Use local models for sensitive data or when minimizing cloud costs and latency.
• Cache model comparisons and expensive prompts to avoid repeated costs.

Example use cases

• Compare GPT-4 and Claude answers to decide which model fits a customer support workflow.
• Estimate token usage and cost for a batch of prompts before running a large job.
• Switch to a local Llama-based model for on-prem inference to meet privacy requirements.
• Stream completions into a chat interface for a live assistant with low latency.
• Run automated A/B tests across multiple providers to evaluate hallucination rates.

FAQ