Caching — ML Junction docs

Public model and route endpoints return Cache-Control: public, max-age=60, stale-while-revalidate=300. Their metrics are rolling aggregates, so clients should revalidate rather than treating them as immutable.

Data · Recommended client behavior
Model/provider catalog · Cache for about 60 seconds and revalidate in the background
Route pricing/capabilities · Use the public cache header; refresh before critical route decisions
Popularity and telemetry · Treat as rolling snapshots, not billing records
Status graphs · Poll according to UI freshness needs; avoid sub-second polling
Inference responses · Cache only when application semantics, privacy, and user isolation permit it
Streaming responses · Never intermediary-cache

Provider prompt caching and HTTP response caching are different systems. Prompt-cache evidence appears in the request receipt and token usage. A continuation token can improve the chance of returning to the same provider surface, but it does not reserve a provider cache entry or keep that cache alive. Expired or unavailable continuation state falls back to visible history and normal routing.

Canonical URL: https://mljunction.com/docs/caching