openrouter-rate-limits skill

Overview

This skill teaches practical strategies to handle OpenRouter rate limits and prevent 429 errors. It focuses on implementing exponential backoff, token-bucket throttling, and request queuing for high-throughput systems. The guidance is language-agnostic but includes Python-ready patterns and testing tips.

How this skill works

The skill inspects request responses from OpenRouter for rate-limit signals (HTTP 429 and Retry-After headers) and applies progressively stronger mitigation: immediate retries with jitter, exponential backoff, and client-side throttling using a token-bucket algorithm. It also outlines request queuing and prioritization so critical calls succeed while bulk work is paced. Configuration points (max retries, base delay, bucket capacity, and refill rate) are highlighted for adaptation to your environment.

When to use it

• You receive 429 or throttling errors from OpenRouter.
• Building services that make many concurrent API calls to OpenRouter.
• Running batch jobs or pipelines that spike request volume.
• Implementing client libraries or SDKs that call OpenRouter on behalf of users.
• Hardening production systems to avoid cascading failures during outages.

Best practices

• Read Retry-After and respect server-provided backoff hints when present.
• Use exponential backoff with randomized jitter to avoid synchronized retries.
• Apply a token-bucket or leaky-bucket limiter to client-side callers to smooth bursts.
• Limit max retries and surface clear errors to callers when limits are exceeded.
• Instrument and log rate-limit events, retry counts, and latency for monitoring and alerts.
• Test backoff and throttling under load and with simulated 429 responses before production rollout.

Example use cases

• API client that automatically pauses and retries when OpenRouter responds with 429.
• High-throughput ingestion pipeline that uses a token-bucket to keep sustained request rate below threshold.
• Web app that prioritizes interactive user requests while queuing background model calls.
• CI batch job that staggers model queries with configurable backoff and retry limits.
• Developer SDK that exposes configurable retry/backoff settings to integrators.

FAQ