Production Readiness Checklist

Once you've chosen a deployment option, use this page to harden your integration before it carries production traffic. It covers error handling and retries, timeouts and cold starts, rate limits, and replica sizing for Dedicated Deployments.

HTTP errors and retries

Roboflow's inference and management APIs use standard HTTP status codes. The full cross-tool table (REST status codes, SDK exceptions, and CLI exit codes) lives in Errors and Status Codes. The key rule for production is to retry only what's actually retryable:

Backoff pattern. For 429 and 5xx, retry with exponential backoff and jitter (for example, 1s, 2s, 4s, 8s with a random offset), capping the number of attempts. Never retry 401/403/404/400 automatically - surface those to your application instead.

When you call the Dedicated Deployments management service (https://roboflow.cloud), check the response code explicitly: a 200 returns a JSON body, and any other code returns an error message as a string.

Timeouts and cold starts

The Serverless Cloud API loads models on demand. The first request to a model that isn't already resident on a server ("warmup") can take several seconds, and a model that has been idle (for example, ~10 minutes between inferences) may be unloaded and need to be reloaded on the next call.

• Set generous client timeouts. A client timeout that's tight enough to trip on a cold start will fail requests that would otherwise have succeeded. Allow headroom for warmup on the first request and after idle periods.
• Prime the model. If predictable latency matters, send a warmup request before your latency-sensitive traffic so the model is already cached.
• Watch the response headers. Serverless responses include x-model-cold-start (whether this request paid the load cost) and x-processing-time. Use them to monitor cold-start frequency and processing time. See Serverless pricing for how these headers factor into billing.
• Keep uploads under the limit. The Serverless Cloud API accepts file uploads up to 20 MB; larger images are rejected. Downsize images before sending (the Python SDK does this automatically), which usually doesn't hurt accuracy because images are resized to the model's input size anyway. Batch Processing enforces the same 20 MB per-image limit.

For sustained low latency without cold starts, use Dedicated Deployments or Self-Hosted Inference instead of the shared serverless endpoint.

Rate limits

• Serverless Cloud API. On a 429, slow down and retry with exponential backoff. If you consistently hit limits or need higher throughput, reach out to your enterprise support contact or the Roboflow forum, or move to a Dedicated Deployment.
• Deployment Manager API. The edge-device management endpoints enforce explicit per-endpoint limits and return 429 when exceeded - for example, device logs are limited to 5 requests per minute per IP and 50 per minute globally, and telemetry reads to 60 requests per minute per device with a 10-request burst over 10 seconds. See the Deployment Manager API for the exact limits and error shapes before you build polling into an integration.

When a documented number doesn't exist for your path, treat 429 as the signal to back off rather than assuming a fixed budget, and contact support if you need a higher limit.

Dedicated Deployment replica sizing

When you create a Dedicated Deployment, you can set min_replicas and max_replicas (both default to 1):

• min_replicas is the number of replicas kept running. A higher minimum reduces cold-start latency under bursty load at the cost of always-on capacity.
• max_replicas caps how far the deployment scales out under load. Raise it for higher peak throughput.

For steady traffic, min_replicas and max_replicas of 1 is the simplest starting point; increase max_replicas when a single replica can't keep up with peak load, and raise min_replicas if the first request after a lull is too slow.

Interaction with auto-pause

Dedicated Deployments auto-pause after a period of inactivity - fixed at 1 hour for dev-cpu and dev-gpu types - and resume when you send a request with your API key. A paused deployment isn't serving replicas, so the request that resumes it pays a resume latency.

• Use the persistent prod-cpu / prod-gpu types for production traffic that must always be ready.
• Reserve the ephemeral dev-cpu / dev-gpu types for testing and prototyping - they are also automatically deleted after a few hours.
• If you need billing based on request count rather than uptime, or a custom pause/replica policy, contact sales.

Before you go live

• Retries wrap every outbound call, retrying only 429 and 5xx with backoff.
• Client timeouts are generous enough to absorb serverless cold starts.
• Images are downsized to stay under the 20 MB upload limit.
• API keys use the minimum required scopes and are stored as secrets, not hard-coded.
• For Dedicated Deployments, min_replicas / max_replicas are sized for your load and you've chosen a prod-* type for always-on traffic.
• You monitor deployments - see Model Monitoring - and alert on error-rate and latency regressions.