Roboflow offers several ways to run your models and Workflows, from fully managed cloud APIs to inference on your own hardware. The right choice depends on whether your workload is real-time or bulk, how much latency you can tolerate, where your data is allowed to live, and how much infrastructure you want to manage.
Use the tables to compare options at a glance, then follow the decision flow below to narrow down.
Roboflow Cloud
Roboflow runs the infrastructure. You send data to an API endpoint and get predictions back.
| Serverless Cloud API | Dedicated Deployments | Batch Processing | |
|---|---|---|---|
| Where it runs | Roboflow cloud (serverless.roboflow.com) | Roboflow-managed private cloud servers (*.roboflow.cloud) | Roboflow cloud, with infrastructure auto-provisioned per job |
| GPU | Yes: GPU-accelerated and auto-scaling | CPU (dev-cpu / prod-cpu) or GPU (dev-gpu / prod-gpu) | CPU or GPU, selected per job |
| Latency profile | Real-time, synchronous. The first request to a model that isn't already loaded incurs a warmup of several seconds, then later requests are fast while the model stays cached. | Consistent, dedicated performance. Deployments auto-pause after a period of inactivity (fixed at 1 hour for dev-cpu / dev-gpu) and resume when you send a request. | Asynchronous, not suitable for real-time. Machines are provisioned when resources are available, typically after a few minutes, with no guaranteed exact start time. |
| Pricing | Metered per inference in credits, based on processing time. See roboflow.com/credits and Serverless pricing. | Pay-per-hour, billed in 1-minute intervals: GPU 1 credit/hour, CPU 0.25 credit/hour. Request-based billing available on request. | CPU and GPU rates differ; GPU jobs are faster but more expensive. See the pricing page. |
| Region | Contact sales for the base API. The Video Streaming API supports us, eu, ap. | US-based data centers only. | Contact sales. |
| Plan availability | Contact sales. | Core and Enterprise. See pricing. | Growth and Enterprise. |
| Best for | Getting started quickly and scaling automatically for real-time single-image or Workflow inference, when the deployment device has a persistent internet connection. | Predictable or production workloads, resource isolation, and large models that need GPU acceleration (ex: Florence-2, SAM2). | Cost-effective processing of large volumes of stored images and videos with a Workflow. |
Self-Hosted
You run inference on your own hardware, so data never has to leave your network.
| Self-Hosted Inference | Edge (Deployment Manager) | |
|---|---|---|
| Where it runs | Your own hardware: edge device, on-prem server, or your own cloud (Docker or pip) | Your edge devices, remotely managed by Roboflow |
| GPU | CPU, CUDA GPU, or NVIDIA Jetson | Depends on the device (NVIDIA Jetson, x86 with NVIDIA GPU, or Roboflow-supplied hardware) |
| Latency profile | Local, low latency with no network round-trip; real-time capable on adequate hardware. | Local edge, real-time. Devices need continuous internet access for remote management and monitoring. |
| Pricing | Free and open source: you provide and manage the infrastructure. TensorRT-optimized packages for private models require Enterprise. | Contact sales. |
| Region | Anywhere (your infrastructure). | Your device locations. |
| Plan availability | All plans (open source). TensorRT private-model packages and licensing to run on more than one device require Enterprise. | Enterprise only. |
| Best for | On-prem, edge, air-gapped, or VPC-bound workloads where you want maximum control over environment and latency. | Setting up, deploying, and monitoring a fleet of edge devices at scale. |
Capabilities at a glance
| Serverless | Dedicated | Self-Hosted | |
|---|---|---|---|
| Fine-tuned and pre-trained models | Yes | Yes | Yes |
| Workflows | Yes | Yes | Yes |
| Heavy foundation models (SAM2, Florence-2, PaliGemma) | No | Yes | Yes |
| Cloud-hosted VLMs (GPT, Claude, Gemini blocks) | Yes | Yes | Yes |
| Video streaming | No | Yes | Yes |
| Custom Python blocks and extra dependencies | No | Yes | Yes |
| Runs offline | No | No | Yes |
| Billing | Per call | Hourly | Free plus metered cloud features |
Scale-up behavior differs too: the Serverless Cloud API scales to zero when idle and back up under load with a cold start of a few seconds, while a Dedicated Deployment takes a minute or two to start. Ephemeral dev-cpu and dev-gpu deployments are limited to short sessions and may be evicted when capacity is needed for higher-priority work; the persistent prod-* types have guaranteed capacity and no session limit.
Bring your own cloud
If compliance policies require workloads to stay inside your own infrastructure, you can run Inference on your own AWS, Azure, or GCP account: see Deploy in Your Own Cloud. Billing is the same as self-hosting on an edge device, and you pay your cloud provider for the machine.
Decision flow
- Real-time or bulk? If you need results as data arrives - live video, interactive apps, per-request inference - take the real-time path. If you're processing a large backlog of stored images or videos, use Batch Processing.
- Sync or async? Real-time inference is synchronous: you send a request and get a response. Batch Processing is asynchronous: you submit a job and collect results when it finishes, so don't depend on an exact start time.
- Cloud or on-prem? If your data can leave your network and you want zero infrastructure to manage, use a Roboflow-hosted cloud option. If your workload must stay on-prem, on the edge, or air-gapped, run Self-Hosted Inference.
- Managed or self-managed?
- In the cloud, choose Dedicated Deployments when you need predictable dedicated performance or a pinned GPU type; otherwise the Serverless Cloud API is the simplest default.
- On your own hardware, use Deployment Manager (Enterprise) when you want Roboflow to manage and monitor a fleet of edge devices for you; otherwise manage the Self-Hosted Inference server yourself.
Once you've picked an option, see the Production Readiness Checklist for error handling, retries, rate limits, and replica sizing.
Inference vs other deployment stacks
Roboflow Inference overlaps with several other tools. This table summarizes when another stack is the better fit, and what you give up by choosing it.
Inference servers
| Tool | How it compares | Choose it if |
|---|---|---|
| NVIDIA Triton | A powerhouse for ML experts deploying at scale, focused on extremely optimized pipelines on NVIDIA hardware. It trades simplicity and iteration speed for raw speed, and its model ensembles are more rigid than Workflows. | You are an ML expert on a tightly defined project that values speed on NVIDIA GPUs above all else. |
| Lightning LitServe | Lightweight, customizable, and task-agnostic (NLP, audio, tabular as well as vision), so it is not as feature-rich for computer vision and has no built-in video streaming or model-chaining abstraction. | You are working on general-purpose ML tasks and want a featureful starting point instead of rolling your own server. |
| TensorFlow Serving | Good if you are deeply invested in TensorFlow across modalities. It can be complex to set up and lacks table-stakes features like pre- and post-processing, which often need custom code. | The TensorFlow ecosystem matters a lot to you and you are willing to do the legwork. |
| TorchServe | The PyTorch equivalent, optimized for serving PyTorch models across vision, NLP, tabular data, and audio. Designed for large-scale cloud deployments and light on vision-specific features like video streaming. | You want to scale and customize PyTorch model deployment and do not need vision-specific functionality. |
| FastAPI or Flask | Rolling your own. Inference's own HTTP interface is built on FastAPI, so starting from scratch means reinventing solved problems. | Your main goal is learning the intricacies of building an inference server. |
Edge deployment
| Tool | How it compares | Choose it if |
|---|---|---|
| Edge Impulse | Focused on very low-power edge devices and embedded systems, and uniquely good with microcontrollers. Its TinyML focus makes it less suited to video processing and modern state-of-the-art models, and it has no Workflows equivalent. | You are building an IoT or wearable device that cannot run more powerful models. |
| NVIDIA DeepStream | NVIDIA's platform for highly optimized video pipelines using TensorRT and CUDA, targeting many of the same problems as Inference. It has a steep learning curve, requires familiarity with NVIDIA tooling, and is not open source. | You are an expert willing to invest heavily in optimizing a single project where throughput is the primary objective. |