Abstract

We present a novel spatiotemporal saliency model for object detection in videos. In contrast to previous methods focusing on exploiting or incorporating different saliency cues, the proposed method aims to use object signatures which can be identified by any kinds of object segmentation methods. We integrate two distinctive saliency maps, which are respectively computed from object proposals of an appearance-dominated method and a motion-dominated algorithm, to obtain a refined spatiotemporal saliency maps. This enables the method to achieve good robustness and precision in identifying salient objects in videos under various challenging conditions. First, an improved appearance-based and a modified motion-based segmentation approaches are separately utilized to extract two kinds of candidate foreground objects. Second, with these captured object signatures, we design a new approach to filter the extracted noisy object pixels and label foreground superpixels in each object signature channel. Third, we introduce a foreground connectivity saliency measure to compute two types of saliency maps, from which an adaptive fusion strategy is exploited to obtain the final spatiotemporal saliency maps for salient object detection in a video. Both quantitative and qualitative experiments on several challenging video benchmarks demonstrate that the proposed method outperforms existing state-of-the-art approaches.

Original languageEnglish (US)
Pages (from-to)285-299
Number of pages15
JournalPattern Recognition
Volume72
DOIs
StatePublished - Dec 1 2017

Fingerprint

Labels
Fusion reactions
Pixels
Object detection
Experiments

Keywords

  • Fusion
  • Object signatures
  • Salient video object detection
  • Spatiotemporal saliency computation

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence

Cite this

Fusing disparate object signatures for salient object detection in video. / Tu, Zhigang; Guo, Zuwei; Xie, Wei; Yan, Mengjia; Veltkamp, Remco C.; Li, Baoxin; Yuan, Junsong.

In: Pattern Recognition, Vol. 72, 01.12.2017, p. 285-299.

Research output: Contribution to journalArticle

Tu, Zhigang ; Guo, Zuwei ; Xie, Wei ; Yan, Mengjia ; Veltkamp, Remco C. ; Li, Baoxin ; Yuan, Junsong. / Fusing disparate object signatures for salient object detection in video. In: Pattern Recognition. 2017 ; Vol. 72. pp. 285-299.
@article{01eeff52f16c42b4a46a007927f60b45,
title = "Fusing disparate object signatures for salient object detection in video",
abstract = "We present a novel spatiotemporal saliency model for object detection in videos. In contrast to previous methods focusing on exploiting or incorporating different saliency cues, the proposed method aims to use object signatures which can be identified by any kinds of object segmentation methods. We integrate two distinctive saliency maps, which are respectively computed from object proposals of an appearance-dominated method and a motion-dominated algorithm, to obtain a refined spatiotemporal saliency maps. This enables the method to achieve good robustness and precision in identifying salient objects in videos under various challenging conditions. First, an improved appearance-based and a modified motion-based segmentation approaches are separately utilized to extract two kinds of candidate foreground objects. Second, with these captured object signatures, we design a new approach to filter the extracted noisy object pixels and label foreground superpixels in each object signature channel. Third, we introduce a foreground connectivity saliency measure to compute two types of saliency maps, from which an adaptive fusion strategy is exploited to obtain the final spatiotemporal saliency maps for salient object detection in a video. Both quantitative and qualitative experiments on several challenging video benchmarks demonstrate that the proposed method outperforms existing state-of-the-art approaches.",
keywords = "Fusion, Object signatures, Salient video object detection, Spatiotemporal saliency computation",
author = "Zhigang Tu and Zuwei Guo and Wei Xie and Mengjia Yan and Veltkamp, {Remco C.} and Baoxin Li and Junsong Yuan",
year = "2017",
month = "12",
day = "1",
doi = "10.1016/j.patcog.2017.07.028",
language = "English (US)",
volume = "72",
pages = "285--299",
journal = "Pattern Recognition",
issn = "0031-3203",
publisher = "Elsevier Limited",

}

TY - JOUR

T1 - Fusing disparate object signatures for salient object detection in video

AU - Tu, Zhigang

AU - Guo, Zuwei

AU - Xie, Wei

AU - Yan, Mengjia

AU - Veltkamp, Remco C.

AU - Li, Baoxin

AU - Yuan, Junsong

PY - 2017/12/1

Y1 - 2017/12/1

N2 - We present a novel spatiotemporal saliency model for object detection in videos. In contrast to previous methods focusing on exploiting or incorporating different saliency cues, the proposed method aims to use object signatures which can be identified by any kinds of object segmentation methods. We integrate two distinctive saliency maps, which are respectively computed from object proposals of an appearance-dominated method and a motion-dominated algorithm, to obtain a refined spatiotemporal saliency maps. This enables the method to achieve good robustness and precision in identifying salient objects in videos under various challenging conditions. First, an improved appearance-based and a modified motion-based segmentation approaches are separately utilized to extract two kinds of candidate foreground objects. Second, with these captured object signatures, we design a new approach to filter the extracted noisy object pixels and label foreground superpixels in each object signature channel. Third, we introduce a foreground connectivity saliency measure to compute two types of saliency maps, from which an adaptive fusion strategy is exploited to obtain the final spatiotemporal saliency maps for salient object detection in a video. Both quantitative and qualitative experiments on several challenging video benchmarks demonstrate that the proposed method outperforms existing state-of-the-art approaches.

AB - We present a novel spatiotemporal saliency model for object detection in videos. In contrast to previous methods focusing on exploiting or incorporating different saliency cues, the proposed method aims to use object signatures which can be identified by any kinds of object segmentation methods. We integrate two distinctive saliency maps, which are respectively computed from object proposals of an appearance-dominated method and a motion-dominated algorithm, to obtain a refined spatiotemporal saliency maps. This enables the method to achieve good robustness and precision in identifying salient objects in videos under various challenging conditions. First, an improved appearance-based and a modified motion-based segmentation approaches are separately utilized to extract two kinds of candidate foreground objects. Second, with these captured object signatures, we design a new approach to filter the extracted noisy object pixels and label foreground superpixels in each object signature channel. Third, we introduce a foreground connectivity saliency measure to compute two types of saliency maps, from which an adaptive fusion strategy is exploited to obtain the final spatiotemporal saliency maps for salient object detection in a video. Both quantitative and qualitative experiments on several challenging video benchmarks demonstrate that the proposed method outperforms existing state-of-the-art approaches.

KW - Fusion

KW - Object signatures

KW - Salient video object detection

KW - Spatiotemporal saliency computation

UR - http://www.scopus.com/inward/record.url?scp=85027494816&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85027494816&partnerID=8YFLogxK

U2 - 10.1016/j.patcog.2017.07.028

DO - 10.1016/j.patcog.2017.07.028

M3 - Article

AN - SCOPUS:85027494816

VL - 72

SP - 285

EP - 299

JO - Pattern Recognition

JF - Pattern Recognition

SN - 0031-3203

ER -