Inference Server Environment Variables

Environment variables that control a self-hosted Roboflow Inference server - execution providers, caching, Workflows, Roboflow API retries, telemetry, HTTPS, and security.

Inference server behavior is controlled by a set of environment variables. Every variable is defined in inference/core/env.py; the ones below are the variables that need more explanation.

Pass them to the container with -e (see Docker configuration options for the most commonly changed settings):

docker run -it --rm -e ENV_VAR_NAME=env_var_value -p 9001:9001 roboflow/roboflow-inference-server-cpu:latest

These variables configure the Inference server. The Roboflow CLI and Python SDK read a different, smaller set of variables (ROBOFLOW_API_KEY, ROBOFLOW_CONFIG_DIR, and others): see Environment Variables in the Reference section.

Execution and models

Variable	Description	Default
`ONNXRUNTIME_EXECUTION_PROVIDERS`	List of execution providers in priority order. A warning is displayed if a provider is not supported on your platform.	See `env.py`
`RUNS_ON_JETSON`	Whether Inference runs on a Jetson device. Set to `True` in all Docker builds for the Jetson architecture.	`False`
`MODEL_VALIDATION_DISABLED`	Makes model loading faster by skipping the trial inference.	`False`
`SAM2_MAX_EMBEDDING_CACHE_SIZE`	Number of SAM2 embeddings held in GPU memory. Each embedding takes 16777216 bytes.	`100`
`SAM2_MAX_LOGITS_CACHE_SIZE`	Number of SAM2 logits held in CPU memory. Each logit takes 262144 bytes.	`1000`
`DISABLE_SAM2_LOGITS_CACHE`	Disables caching of SAM2 logits. Useful for debugging or to minimize memory usage, at the cost of slower repeated similar requests.	`False`

`inference-models` backend

Variable	Description	Default
`USE_INFERENCE_MODELS`	Selects the `inference-models` backend.	`False`
`MAX_INFERENCE_MODELS_CACHE_SIZE_MB`	Enables the `inference-models` cache watchdog. When set above `0`, the watchdog prunes model artifacts (oldest and biggest first) to prevent the system running out of disk space over time. Only applies when `USE_INFERENCE_MODELS=True`.	`-1`
`INFERENCE_MODELS_CACHE_WATCHDOG_INTERVAL_MINUTES`	Frequency of `inference-models` cache watchdog cycles. The minimum is 15 minutes.	`60`
`ENABLE_CUDA_MEMORY_RECLAMATION_WATCHDOG`	Enables a background daemon that periodically returns cached-but-unused CUDA memory to the driver via `torch.cuda.empty_cache()`. PyTorch's caching allocator retains freed device blocks in its own pool and never releases them on its own, so on a long-running server the high-water mark of concurrent or batched inference is sticky and reserved VRAM only grows. This watchdog reclaims that slack on a fixed interval; live allocations are untouched. It does not prevent an OOM caused by a genuinely oversubscribed concurrent peak. Only meaningful with `USE_INFERENCE_MODELS=True` on CUDA.	`False`
`CUDA_MEMORY_RECLAMATION_WATCHDOG_INTERVAL_SECONDS`	Interval between reclamation cycles of the CUDA memory watchdog. The minimum is `5` seconds (lower values are clamped up). Only applies when `ENABLE_CUDA_MEMORY_RECLAMATION_WATCHDOG=True`.	`300`

Workflows and video

Variable	Description	Default
`ENABLE_WORKFLOWS_PROFILING`	Allows the server to return Workflows profiler traces to the client.	`False`
`WORKFLOWS_PROFILER_BUFFER_SIZE`	Size of the profiler buffer: the number of consecutive Workflows Execution Engine `run(...)` invocations traced in the buffer.	`64`
`WORKFLOWS_DEFINITION_CACHE_EXPIRY`	Number of seconds to cache Workflow definitions returned by `get_workflow_specification(...)`.	`900` (15 minutes)
`ENABLE_STREAM_API`	Enables the video management API. The standard CPU, GPU, and TensorRT images set this to `True`, as do the JetPack 5.1.1 and 6.2.0 images; slim images and the JetPack 6.0.0 and 7.1.0 images do not.	`False` outside the images that set it
`STREAM_API_PRELOADED_PROCESSES`	How many idle video workers are warmed up. This reduces worker start time on GPU.	`0`

GPU tensor pipeline

On a capable NVIDIA GPU, Workflows can keep image data on the GPU from video decode through model inference to output, instead of moving it through the CPU. This lowers latency for GPU-heavy video Workflows. It needs USE_INFERENCE_MODELS=True, a GPU with compute capability 7.5 or higher (ex: T4, RTX 20-series and newer; not V100), and the onnx.gpu or Jetson Docker image. Without a matching GPU and image, Inference falls back to normal CPU-based execution.

Variable	Description	Default
`ENABLE_TENSOR_DATA_REPRESENTATION`	Turns on the GPU tensor pipeline: GPU-resident Workflows execution and hardware-accelerated (NVDEC) video decoding.	`False`
`WORKFLOWS_IMAGE_TENSOR_DEVICE`	Torch device Workflow tensors are placed on when the tensor pipeline is enabled.	`cuda` if available, else `cpu`
`VIDEO_SOURCE_BUFFER_SIZE`	Number of decoded frames buffered per video source.	`8` when the tensor pipeline is enabled, `64` otherwise
`VIDEO_SOURCE_ADAPTIVE_BACKPRESSURE`	Drops buffered frames based on buffer state instead of estimated frame rate.	Matches `ENABLE_TENSOR_DATA_REPRESENTATION`
`WORKFLOWS_ENFORCE_DENSE_INSTANCE_MASKS`	Returns dense instance segmentation masks instead of RLE-encoded ones from tensor pipeline models.	`False`
`WORKFLOWS_SAM_VIDEO_MASK_REPRESENTATION`	Mask format (`rle` or `dense`) used by SAM video tracking blocks in the tensor pipeline.	`rle`
`DISABLE_GSTREAMER_VIDEO_SOURCES`	Disables GStreamer-based video sources, forcing standard CPU decoding.	`False`

Roboflow API connectivity

Variable	Description	Default
`TRANSIENT_ROBOFLOW_API_ERRORS`	Comma-separated list of HTTP codes from the Roboflow API that should be retried (GET endpoints only).	Not set
`TRANSIENT_ROBOFLOW_API_ERRORS_RETRIES`	Number of times transient errors (connection errors and transient HTTP codes) are retried (GET endpoints only).	`3`
`TRANSIENT_ROBOFLOW_API_ERRORS_RETRY_INTERVAL`	Delay between retries of transient Roboflow API errors (GET endpoints only).	`3`
`RETRY_CONNECTION_ERRORS_TO_ROBOFLOW_API`	Whether connection errors to the Roboflow API should be retried (GET endpoints only).	`False`
`ROBOFLOW_API_REQUEST_TIMEOUT`	Timeout in seconds (integer) for requests to the Roboflow API.	Not set
`API_PROXY_BASE_URL`	Base URL used for Roboflow API proxy requests to `apiproxy/*` endpoints. Set this to a direct heavy-API Cloud Run service root to bypass Firebase Hosting timeouts for long-running third-party proxy calls.	Value of `API_BASE_URL`
`DISABLE_VERSION_CHECK`	Disables the Inference version check that runs in a background thread. Force-set to `True` (overriding an explicit `False`) when `SECURE_GATEWAY` is configured, because `api.github.com` is unreachable behind the gateway.	`False`
`SECURE_GATEWAY`	Address of a Roboflow Secure Gateway for air-gapped deployments (legacy alias: `LICENSE_SERVER`). Routes Roboflow API and model download traffic through the gateway proxy, force-disables the version check, and falls back to local Workflow step execution when `remote` plus `hosted` is configured. See Docker configuration options.	Not set

Monitoring and telemetry

Variable	Description	Default
`ENABLE_PROMETHEUS`	Enables the Prometheus `/metrics` endpoint. See Telemetry.	`True` for the Docker Hub images
`DOCKER_SOCKET_PATH`	Path to the Docker daemon socket mounted into the container. When provided, enables polling Docker container stats from the daemon socket. See Telemetry.	Not set
`METRICS_ENABLED`	Controls Roboflow Model Monitoring.	`True`
`MODEL_MONITORING_CACHE_BACKEND`	Cache backend for model-monitoring pingback data. Use `default` to follow the normal cache selection (Redis when `REDIS_HOST` is configured, otherwise memory), or `memory` to force process-local buffering and keep Redis off the inference hot path.	`default`

HTTPS

Variable	Description	Default
`ENABLE_HTTPS`	Toggles HTTPS for the Inference server. When `True`, the server reads `SSL_CERTFILE` and `SSL_KEYFILE` and serves traffic over TLS. See Serving Inference over HTTPS.	`False`
`SSL_CERTFILE`	Path to a PEM-encoded TLS certificate served when `ENABLE_HTTPS=True`.	`/etc/inference/certs/server.crt`
`SSL_KEYFILE`	Path to the PEM-encoded TLS private key paired with `SSL_CERTFILE`.	`/etc/inference/certs/server.key`
`SSL_KEYFILE_PASSWORD`	Passphrase used to decrypt `SSL_KEYFILE` when the private key is encrypted.	Not set
`SSL_CA_CERTS`	Path to a CA bundle used when client certificate verification (mTLS) is required.	Not set

Authentication and input security

Variable	Description	Default
`WORKSPACES_WHITELISTED_FOR_LOCAL_DEPLOYMENT`	Comma-separated list of Roboflow workspace URL slugs allowed to execute requests against this server. When set, every request (apart from the docs, landing page, and health, liveness, and metrics endpoints) is authorized with a Roboflow API key.	Not set

Variables that control custom Python execution and URL image fetching (ALLOW_CUSTOM_PYTHON_EXECUTION_IN_WORKFLOWS, ALLOW_URL_INPUT, ALLOW_URL_TO_NON_GLOBAL_ADDRESSES, VALIDATE_IMAGE_URL_REDIRECTS, and others) are documented in Securing a Self-Hosted Server and Accepted Input Formats.