ProxyDet: Synthesizing Proxy Novel Classes via Classwise Mixup for Open-Vocabulary Object Detection

Joonhyun Jeong1,2, Geondo Park2, Jayeon Yoo3, Hyungsik Jung1, Heesu Kim1
1NAVER Cloud, ImageVision, 2KAIST, 3Seoul National University
Paper Video Code Demo

ProxyDet can synthesize any novel categories and detect novel classes with high accuracy. ProxyDet is a simple yet effective training technique that can be easily added-on any open-vocabulary object detection frameworks, without considerable computation overhead.

Highlight

Motivational observation

  • Empirically, we found that numerous novel (unseen) classes reside within convex hull of base (seen) classes.
  • We synthesize and train proxy classes for the novel classes via our class-wise mixup strategy of base class representations.
    • The avantages of ProxyDet are summarized as follows:
  • Awareness of novel classes. ProxyDet exhibits high proximity to the novel classes.
  • Simple add-on training auxiliary. ProxyDet provides a simple technique that can be applied on any open-vocab detection frameworks.
  • Performance improvement. While its simplicity, ProxyDet offers remarkable performance improvement on novel classes. The newly-introduced proxy-novel classes help expanding vocabulary.

    • Superior detection ability

    ProxyDet can detect novel classes with higher accuarcy than other models.

  • ProxyDet showcases impressive novel class performance, outperforming fully-supervised detector.
  • Adding our method on Detic brings large improvement on various backbones.

    • BibTeX

    @article{jeong2023proxydet,
        title={ProxyDet: Synthesizing Proxy Novel Classes via Classwise Mixup for Open Vocabulary Object Detection},
        author={Jeong, Joonhyun and Park, Geondo and Yoo, Jayeon and Jung, Hyungsik and Kim, Heesu},
        journal={arXiv preprint arXiv:2312.07266},
        year={2023}
      }