Predictions are correct on CPU, wrong on GPU #460
Description
I have an instance segmentation model (RTMDet-Tiny) trained and exported from Geti 2.13. When using this model to generate predictions with Model API / OpenVINO, I discovered that the results vary wildly between different devices, and they are plain wrong when I run inference on GPU (iGPU in my case).
Input image
CPU predictions (good)
GPU predictions (bad)
Steps to Reproduce
- Download and extract the archive with the model, sample image and script
- Install the requirements
- Run the sample script (
main.py, source code below), optionally changing theINFERENCE_DEVICE
import cv2
import numpy as np
from model_api.models import Model
from model_api.models import MaskRCNNModel
from model_api.models.result import InstanceSegmentationResult
from model_api.visualizer import BoundingBox, Flatten, Polygon, Label
from model_api.visualizer.scene import InstanceSegmentationScene
from PIL import Image
from typing import Any
IMAGE_PATH = "20251216_150128_215-original.jpg"
MODEL_PATH = "model.xml"
INFERENCE_DEVICE = "CPU" # or 'GPU'
def main():
# Load the image with OpenCV as numpy array
image = cv2.imread(IMAGE_PATH)
if image is None:
print(f"Error: Unable to load image at {IMAGE_PATH}")
return
# Switch BGR to RGB
image_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
# Load the model with Model API
mapi_model: MaskRCNNModel = Model.create_model(model=MODEL_PATH, device=INFERENCE_DEVICE)
# Run inference
# Option 1: using the callable interface
result: InstanceSegmentationResult = mapi_model(image_rgb)
# Save the results to file
image_pil = Image.fromarray(image)
segmentation_scene = InstanceSegmentationScene(
image=image_pil, result=result, layout=Flatten(BoundingBox, Polygon, Label)
)
rendered_segmentation_pil = segmentation_scene.render()
rendered_segmentation_np = np.array(rendered_segmentation_pil)
cv2.imwrite("segmentation_result.png", rendered_segmentation_np)
print("Segmentation result saved to segmentation_result.png")
if __name__ == "__main__":
main()Environment
- OS: Ubuntu 24.04
- OpenVINO version: 2025.3
- Model API version: 0.3.0.4
- Inference device model and memory: Intel(R) Core(TM) Ultra 7 265U, 64GB RAM, iGPU