Roboflow Docs

Configuration

Configure InferenceHTTPClient defaults, per-task model parameters, and Workflows execution options with InferenceConfiguration.

Configuration options

Configuring with context managers

The methods use_configuration(...) and use_model(...) are designed to work in context managers. Once the context manager is left, old config values are restored.

from inference_sdk import InferenceHTTPClient, InferenceConfiguration

image_url = "https://source.roboflow.com/pwYAXv9BTpqLyFfgQoPZ/u48G0UpWfk8giSw7wrU8/original.jpg"

custom_configuration = InferenceConfiguration(confidence_threshold=0.8)
# Replace ROBOFLOW_API_KEY with your Roboflow API Key
CLIENT = InferenceHTTPClient(
    api_url="http://localhost:9001",
    api_key="ROBOFLOW_API_KEY"
)

with CLIENT.use_configuration(custom_configuration):
    _ = CLIENT.infer(image_url, model_id="soccer-players-5fuqs/1")

with CLIENT.use_model("soccer-players-5fuqs/1"):
    _ = CLIENT.infer(image_url)

# after leaving context manager - changes are reverted and `model_id` is still required
_ = CLIENT.infer(image_url, model_id="soccer-players-5fuqs/1")

As you can see, model_id is required for a prediction method only when a default model is not configured.

The model ID is composed of the string <project_id>/<version_id>. See Workspace and Project IDs to find these pieces of information.

Setting the configuration once and using it until the next change

The methods configure(...) and select_model(...) alter the client state and the change is preserved until the next change.

from inference_sdk import InferenceHTTPClient, InferenceConfiguration

image_url = "https://source.roboflow.com/pwYAXv9BTpqLyFfgQoPZ/u48G0UpWfk8giSw7wrU8/original.jpg"

custom_configuration = InferenceConfiguration(confidence_threshold=0.8)
# Replace ROBOFLOW_API_KEY with your Roboflow API Key
CLIENT = InferenceHTTPClient(
    api_url="http://localhost:9001",
    api_key="ROBOFLOW_API_KEY"
)

CLIENT.configure(custom_configuration)
CLIENT.infer(image_url, model_id="soccer-players-5fuqs/1")

# custom configuration still holds
CLIENT.select_model(model_id="soccer-players-5fuqs/1")
_ = CLIENT.infer(image_url)

# custom configuration and selected model - still holds
_ = CLIENT.infer(image_url)

You may also initialise in chain mode:

from inference_sdk import InferenceHTTPClient, InferenceConfiguration

# Replace ROBOFLOW_API_KEY with your Roboflow API Key
CLIENT = InferenceHTTPClient(api_url="http://localhost:9001", api_key="ROBOFLOW_API_KEY") \
    .select_model("soccer-players-5fuqs/1")

Overriding model_id for a specific call

model_id can be overridden for a specific call:

from inference_sdk import InferenceHTTPClient

image_url = "https://source.roboflow.com/pwYAXv9BTpqLyFfgQoPZ/u48G0UpWfk8giSw7wrU8/original.jpg"

# Replace ROBOFLOW_API_KEY with your Roboflow API Key
CLIENT = InferenceHTTPClient(api_url="http://localhost:9001", api_key="ROBOFLOW_API_KEY") \
    .select_model("soccer-players-5fuqs/1")

_ = CLIENT.infer(image_url, model_id="another-model/1")

Details about client configuration

InferenceHTTPClient provides the InferenceConfiguration dataclass to hold the full configuration.

from inference_sdk import InferenceConfiguration

Overriding fields in this config changes the behaviour of the client (and of the API serving the model). Specific fields are used in specific contexts. In particular:

Classification model

  • visualize_predictions: flag to enable / disable visualisation
  • confidence_threshold as confidence
  • stroke_width: width of stroke in visualisation
  • disable_preproc_auto_orientation, disable_preproc_contrast, disable_preproc_grayscale, disable_preproc_static_crop to alter server-side pre-processing
  • disable_active_learning to prevent the Active Learning feature from registering the datapoint (can be useful, for instance, while testing a model)
  • active_learning_target_dataset - when making inference from a specific model (let's say project_a/1) and you want to save data in another project project_b, the latter should be pointed to by this parameter. Note that you cannot use different types of models in project_a and project_b; if that is the case, data will not be registered.
  • source: optional string that sets a "source" attribute on the inference call. If using model monitoring, this is logged with the inference request so you can filter or query inference requests coming from a particular source, for example to identify which application, system, or deployment is making the request.
  • source_info: optional string that sets an additional "source_info" attribute on the inference call, for example to identify a sub-component in an app.

Object detection model

  • visualize_predictions: flag to enable / disable visualisation
  • visualize_labels: flag to enable / disable label visualisation if visualisation is enabled
  • confidence_threshold as confidence
  • class_filter to filter out a list of classes
  • class_agnostic_nms: flag to control whether NMS is class-agnostic
  • fix_batch_size
  • iou_threshold: to dictate the NMS IoU threshold
  • stroke_width: width of stroke in visualisation
  • max_detections: max detections to return from the model
  • max_candidates: max candidates for post-processing from the model
  • disable_preproc_auto_orientation, disable_preproc_contrast, disable_preproc_grayscale, disable_preproc_static_crop to alter server-side pre-processing
  • disable_active_learning, active_learning_target_dataset, source, source_info - as described above

Keypoint detection model

  • visualize_predictions: flag to enable / disable visualisation
  • visualize_labels: flag to enable / disable label visualisation if visualisation is enabled
  • confidence_threshold as confidence
  • keypoint_confidence_threshold (as keypoint_confidence) to filter out detected keypoints based on model confidence
  • class_filter to filter out a list of object classes
  • class_agnostic_nms: flag to control whether NMS is class-agnostic
  • fix_batch_size
  • iou_threshold: to dictate the NMS IoU threshold
  • stroke_width: width of stroke in visualisation
  • max_detections: max detections to return from the model
  • max_candidates: max candidates for post-processing from the model
  • disable_preproc_auto_orientation, disable_preproc_contrast, disable_preproc_grayscale, disable_preproc_static_crop to alter server-side pre-processing
  • disable_active_learning, active_learning_target_dataset, source, source_info - as described above

Instance segmentation model

  • visualize_predictions: flag to enable / disable visualisation
  • visualize_labels: flag to enable / disable label visualisation if visualisation is enabled
  • confidence_threshold as confidence
  • class_filter to filter out a list of classes
  • class_agnostic_nms: flag to control whether NMS is class-agnostic
  • fix_batch_size
  • iou_threshold: to dictate the NMS IoU threshold
  • stroke_width: width of stroke in visualisation
  • max_detections: max detections to return from the model
  • max_candidates: max candidates for post-processing from the model
  • disable_preproc_auto_orientation, disable_preproc_contrast, disable_preproc_grayscale, disable_preproc_static_crop to alter server-side pre-processing
  • mask_decode_mode
  • tradeoff_factor
  • disable_active_learning, active_learning_target_dataset, source, source_info - as described above

Configuration of the client

  • output_visualisation_format: one of VisualisationResponseFormat.BASE64, VisualisationResponseFormat.NUMPY, VisualisationResponseFormat.PILLOW. Given that server-side visualisation is enabled, you may choose which format should be used in the output.
  • client_downsizing_disabled: set to False if you want to perform client-side downsizing. Default True. Client-side scaling is only supposed to down-scale (keeping aspect ratio) the input for inference, to utilise the internet connection more efficiently (at the price of image manipulation / transcoding). Model input size information is used to determine the target size; if not available, default_max_input_size is used.
  • max_concurrent_requests: max number of concurrent requests that can be started
  • max_batch_size: max number of elements that can be injected into a single request
  • workflow_run_retries_enabled: flag that decides if transient errors in Workflows executions should be retried. Defaults to true and the default can be altered with the environment variable WORKFLOW_RUN_RETRIES_ENABLED.

Configuration of Workflows execution

  • profiling_directory: specifies the location where Workflows profiler traces are saved. By default, it is the ./inference_profiling directory.