Set idempotency_key for important non-streaming operations. The key is scoped to the gateway API key. A completed result is cached for one hour, and a concurrent duplicate returns a conflict instead of executing twice.
Generate one stable idempotency key per logical operation, not per HTTP retry.. Do not reuse a key for different prompts or users.. Retry validation and authorization failures only after changing the request or policy.. Retry network failures, rate limits, and retryable provider errors with exponential backoff and jitter.. Streaming retries require application-level handling because partial content may already be visible.