Model Weights Download

When deploying Roboflow Inference, model weights are downloaded to your device where inference runs locally. This page covers how the inference package downloads and caches those weights.

Overview

Model weights are downloaded automatically the first time you run inference with a given model. The weights are cached locally on your device, and all inference is performed on-device (not in the cloud).

How it works:

1. Download model weights to your device while connected to the internet.
2. Weights are cached locally on your machine.
3. Run inference on-device using the cached weights.

This approach works across all Roboflow deployment methods and ensures fast, local inference.

Cache location

By default, model weights are cached in /tmp/cache. This directory is cleared on system reboot, which means you need to re-download model weights after each restart.

For production deployments or any scenario where you need weights to persist across reboots, you must configure a persistent cache directory using the MODEL_CACHE_DIR environment variable:

import os
# Set to a persistent directory (not /tmp)
os.environ["MODEL_CACHE_DIR"] = "/home/user/.roboflow/cache"

from inference import get_model
# ... rest of your code

Alternatively, set it system-wide:

export MODEL_CACHE_DIR="/home/user/.roboflow/cache"

Make sure the directory exists and has appropriate permissions:

mkdir -p /home/user/.roboflow/cache
chmod 755 /home/user/.roboflow/cache

Native Python API

The native Python API automatically downloads and caches weights when you load a model with get_model().

Pre-downloading weights

from inference import get_model

# Load model (downloads and caches weights)
model = get_model(
    model_id="rfdetr-base",
    api_key="YOUR_ROBOFLOW_API_KEY"
)
print("Model weights cached!")

Running inference

from inference import get_model

# Uses cached weights for on-device inference
model = get_model(
    model_id="rfdetr-base",
    api_key="YOUR_ROBOFLOW_API_KEY"
)

results = model.infer("path/to/image.jpg")

On a self-hosted Inference Server, you can pre-load weights over HTTP instead: see Model Management.

Best practices

1. Configure a persistent cache first. Before downloading any weights, configure MODEL_CACHE_DIR to point to a persistent directory (not /tmp). This is essential for production deployments to avoid losing cached weights on reboot.
2. Pre-download during setup. Download all required model weights during your deployment setup phase to ensure they are cached and ready.
3. Use a persistent cache in Docker. Always mount a persistent volume when running in Docker containers. Weights stored in the container filesystem are lost on restart.
4. Verify before deployment. Verify that models are properly cached and that the cache directory persists across reboots before deploying to production.
5. Document model IDs. Keep a list of all model IDs and versions your application requires for easier pre-caching and troubleshooting.
6. Consider storage. Model weights can be large (100MB to 1GB+ per model). Ensure sufficient disk space is available in your persistent cache directory.
7. Test reboot behavior. After caching weights, test that they persist after a system reboot to ensure your cache configuration is correct.

Troubleshooting

Weights disappear after reboot

The default cache location (/tmp/cache) is cleared on reboot. Configure a persistent cache directory as described in Cache location, or use a persistent volume mount for Docker.

Model not found error

• Verify the model was actually downloaded (check the cache directory with ls -lh $MODEL_CACHE_DIR).
• Ensure you are using the exact same model_id as when downloading.
• Check that MODEL_CACHE_DIR is set correctly if using a custom location.

Permission issues

Ensure the application has read/write permissions to the cache directory:

chmod -R 755 /path/to/cache