CoRef-DETR: Decoupled Global Context and Selective Boundary-Category Refinement for Underwater Instance Segmentation

Underwater instance segmentation supports marine surveys, aquaculture, and underwater robotics. Haze, colour casts, and low contrast blur object boundaries, and recent benchmarks distinguish up to 158 fine-grained categories. The strongest published methods mostly have 219 to 632 million parameters. This paper proposes CoRef-DETR, a compact detection transformer that targets three error sources of a real-time detector: context, boundaries, and categories. First, a decoupled global context (DGC) detector, with the base detector’s 33.4 M parameters, applies global self-attention to a 384-pixel token grid but supervises and decodes masks in a 576-pixel frame. Second, a posterior-conditioned boundary refiner with 47.7 k parameters corrects mask logits on predicted boundaries, and a quality score on the corrected masks re-ranks the detections. Third, a crop-probe category branch on frozen DINOv2 features revises the categories of at most three masks per image, where a rule fixed on validation data chooses soft fusion or a confident-disagreement gate. On five protocols of UIIS, UIIS10K, USIS10K, and USIS16K, CoRef-DETR achieves state-of-the-art mask average precision (AP), exceeding the strongest published results by 2.3 to 5.2 points, and outperforms three reproduced recent methods on the same test images (on USIS16K, with its category branch). In half precision with TensorRT, it processes an image in a median of 34 to 40 ms, JPEG decoding included, on a T4 GPU, 1.4 to 11 times faster than these methods. Ablations attribute most of DGC’s gain to the larger frame; an error analysis locates the remaining error in crowds of small objects.