langfuse-integration skill

Overview

This skill integrates LangFuse LLM observability into your agent workflows for tracing, analytics, and cost tracking. It enables structured tracing of LLM calls, prompt versioning, and custom metadata to support debugging and optimization. The integration supports both cloud and self-hosted LangFuse deployments and common attachment points like LangChain callbacks and direct SDK calls.

How this skill works

It instruments LLM calls, chains, and agents to emit traces that include latency, tokens, and cost metrics. Prompts are versioned and recorded alongside model outputs so you can correlate input changes with quality and cost. The integration exposes configuration for keys, host URL, sampling rate, and custom metadata, and can attach via LangChain callback handlers, decorators, or the LangFuse SDK.

When to use it

• You need end-to-end observability for multi-step agent workflows
• You want to track LLM usage, latency, and token-based costs
• You must version prompts and correlate versions with outcomes
• You plan to run evaluations using labeled datasets and automated scoring
• You want custom dashboards and alerting for anomalous costs or regressions

Best practices

• Adopt consistent and descriptive trace naming across agents and chains
• Include meaningful metadata (user id, dataset, intent) for easier filtering
• Version prompts on every change and store the prompt id in traces
• Set sensible sampling rates to balance cost and signal
• Configure alerts for cost spikes, latency regressions, and quality drops

Example use cases

• Monitor costs and latency across production agents to identify expensive prompts
• Compare prompt versions by correlating prompt id with quality and cost metrics
• Run evaluation datasets and record scores to track model drift over time
• Create dashboards and alerts for high-cost operations or unexpected token usage
• Instrument LangChain-based chains with a callback handler to capture detailed traces

FAQ