|
|
--- |
|
|
tags: |
|
|
- object-detection |
|
|
- instance-segmentation |
|
|
- transformer |
|
|
- detr |
|
|
- npu |
|
|
- qualcomm |
|
|
- roboflow |
|
|
- real-time |
|
|
- edge-deployment |
|
|
license: cc-by-nc-4.0 |
|
|
library_name: nexa-sdk |
|
|
--- |
|
|
|
|
|
# RF-DETR-Seg-Preview-NPU |
|
|
|
|
|
Run RF-DETR-Seg-Preview on Qualcomm NPU with nexaSDK. |
|
|
|
|
|
## Quickstart |
|
|
|
|
|
Install nexaSDK and create a free account at [sdk.nexa.ai](https://sdk.nexa.ai) |
|
|
|
|
|
Activate your device with your access token: |
|
|
|
|
|
```bash |
|
|
nexa config set license '<access_token>' |
|
|
``` |
|
|
|
|
|
Run the model locally in one line: |
|
|
|
|
|
```bash |
|
|
nexa infer NexaAI/rf-detr-seg-preview-npu |
|
|
``` |
|
|
|
|
|
## Model Description |
|
|
|
|
|
RF-DETR-Seg-Preview is a real-time object detection and instance segmentation model developed by Roboflow, based on the Transformer architecture. It is the first real-time model to achieve over 60 AP on the COCO dataset, combining high accuracy with efficient inference performance. |
|
|
|
|
|
The model leverages a DINOv2 visual backbone and a lightweight DETR design, providing excellent transfer learning capabilities and domain adaptability. Its end-to-end architecture eliminates the need for non-maximum suppression (NMS) and anchor boxes, simplifying the detection pipeline. |
|
|
|
|
|
RF-DETR-Seg-Preview brings state-of-the-art detection and segmentation accuracy to real-time applications, making it ideal for edge deployment scenarios where both speed and precision are critical. |
|
|
|
|
|
## Features |
|
|
|
|
|
- **High-Accuracy Real-Time Detection & Segmentation**: Achieves 60.5 mAP on COCO dataset while maintaining real-time performance, suitable for applications requiring both speed and accuracy. |
|
|
|
|
|
- **Domain Adaptability**: Through the DINOv2 backbone network, enables cross-domain transfer learning, suitable for complex scenarios such as aerial imagery and industrial applications. |
|
|
|
|
|
- **Dynamic Resolution Support**: Supports multi-resolution training and inference, allowing precision-speed trade-offs without retraining. |
|
|
|
|
|
- **Efficient Edge Deployment**: Optimized for Qualcomm NPU, providing fast inference and low latency on edge devices with limited resources. |
|
|
|
|
|
- **End-to-End Architecture**: Eliminates the need for NMS and anchor boxes, simplifying the detection and segmentation pipeline. |
|
|
|
|
|
- **Instance Segmentation**: Provides pixel-level segmentation masks for each detected object, enabling precise object boundary identification. |
|
|
|
|
|
## Use Cases |
|
|
|
|
|
- **Real-Time Video Analysis**: Fast object detection and instance segmentation in image or video streams, suitable for autonomous driving, security monitoring, and surveillance systems. |
|
|
|
|
|
- **Edge Device Deployment**: Lightweight design enables deployment on mobile devices, embedded systems, and other edge devices with resource constraints. |
|
|
|
|
|
- **Autonomous Systems**: Detection and segmentation of pedestrians, vehicles, and other objects for autonomous navigation and robotics. |
|
|
|
|
|
- **Custom Dataset Fine-Tuning**: Supports fine-tuning on custom datasets to meet specific application requirements. |
|
|
|
|
|
- **Production Environments**: Efficient deployment in production or research environments requiring real-time performance. |
|
|
|
|
|
## Inputs and Outputs |
|
|
|
|
|
**Input:** |
|
|
|
|
|
- Image path |
|
|
|
|
|
**Output:** |
|
|
|
|
|
- Detection results including object classes, bounding box coordinates, and confidence scores |
|
|
|
|
|
## License |
|
|
|
|
|
All NPU-related components of this project are licensed under the Creative Commons Attribution–NonCommercial 4.0 International (CC BY-NC 4.0) license. |
|
|
|
|
|
Commercial licensing or usage rights must be obtained through a separate agreement. For inquiries regarding commercial use, please contact [email protected] |
|
|
|
