Production Checklist — ML Junction docs

Use a production API key with explicit model, RPM, TPM, concurrency, and output-token limits.. Use routing.preview to verify at least one route satisfies every required capability and privacy rule.. Enable production_mode for deterministic provider/route controls and charge caps.. Set strict_params when silently ignored parameters could change behavior.. Set max_platform_charge_usd and max_byok_fee_usd according to your risk budget.. Preserve privacy and context capacity across fallback.. Use idempotency keys for side-effecting or expensive non-streaming workflows.. Validate structured output and tool arguments in your application as well as at the gateway.. Log request ID, route ID, provider, model, usage, actual charge, latency, and finish reason.. Monitor uptime separately from quality failures.. Alert on p95/p99 latency, availability failures, rate limits, and unexpected fallback frequency.. Evaluate every route and quantization used in production on representative tasks.. Keep provider and gateway credentials in a secret manager and rotate them regularly.. Test cancellation, timeout, rate-limit, malformed-output, and provider-outage paths before launch.. Test continuation with a valid token, no token, expired token, removed route/account, provider outage, model switch, missing reasoning details, tool-result mismatch, and tampered token.

Canonical URL: https://mljunction.com/docs/production-checklist