Multi-View Segmentation

Segmenting objects of interest is a first step for many applications in computer vision such as scene analysis, matting, compositing for post-production, image indexing and 3D reconstruction. There are many situations where various viewpoints on the object are available and we propose here a complete probabilistic framework to account for geometric and appearance cues, allowing foreground/background segmentation without 3D object reconstruction or disparity map estimation. We cast the multi-view segmentation problem as Maximum A Posteriori (MAP) estimation [1, 2] and segmentation estimates are obtained with a classical EM scheme yielding a simple and efficient approach.

Building on this first method, we propose a unified solution [3] dealing with intra-view, inter-view, and temporal cues in a multi-view image and video segmentation context, with a single consistent MRF model. The algorithm has been demonstrated on very chalenging datasets, including MVOS segmentation with videos from moving hand held cameras.

plantResult Results on Plant dataset with only 3 views HPAnimation_small
Segmentation results with hand-held cameras

dancers1_small
Comparison between our proposed approach [3] on the right and Video SnapCut, a monocular approach.

References

[1] [pdf] [doi] A. Djelouah, J. Franco, E. Boyer, F. Le Clerc, and P. Perez, “Sparse Multi-View Consistency for Object Segmentation A,” IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1-14, 2015.
[Bibtex]
@article{djelouah-pami2015,
TITLE = {{Sparse Multi-View Consistency for Object Segmentation A}},
AUTHOR = {Djelouah, Abdelaziz and Franco, Jean-S{\'e}bastien and Boyer, Edmond and Le Clerc, Fran{\c c}ois and Perez, Patrick},
URL = {https://hal.inria.fr/hal-01115557},
JOURNAL = {{IEEE Transactions on Pattern Analysis and Machine Intelligence}},
PUBLISHER = {{Institute of Electrical and Electronics Engineers (IEEE)}},
PAGES = {1 - 14},
YEAR = {2015},
DOI = {10.1109/TPAMI.2014.2385704},
PDF = {https://hal.inria.fr/hal-01115557/file/paper_V3.pdf},
HAL_ID = {hal-01115557},
HAL_VERSION = {v1},
}
[2] [pdf] A. Djelouah, J. Franco, E. Boyer, F. Leclerc, and P. Pérez, “N-Tuple Color Segmentation for Multi-View Silhouette Extraction,” in ECCV’12 – 12th European Conference on Computer Vision, 2012, pp. 818-831.
[Bibtex]
@inproceedings{djelouah-eccv12,
url = {http://hal.inria.fr/hal-00735718},
title = {{N-Tuple Color Segmentation for Multi-View Silhouette Extraction}},
author = {Djelouah, Abdelaziz and Franco, Jean-S{\'e}bastien and Boyer, Edmond and Leclerc, Fran{\c c}ois and P{\'e}rez, Patrick},
affiliation = {Technicolor [Cesson S{\'e}vign{\'e}] , MORPHEO - INRIA Grenoble Rh{\^o}ne-Alpes / LJK Laboratoire Jean Kuntzmann},
booktitle = {{ECCV'12 - 12th European Conference on Computer Vision}},
publisher = {Springer},
pages = {818-831},
year = {2012},
pdf = {http://hal.inria.fr/hal-00735718/PDF/Final\_N-tuple\_Multi-View\_Silhouette\_Extraction-1.pdf},
}
[3] [pdf] A. Djelouah, J. Franco, E. Boyer, F. Le Clerc, and P. Pérez, “Multi-View Object Segmentation in Space and Time,” in ICCV’13 – International Conference On Computer Vision, 2013.
[Bibtex]
@inproceedings{djelouah-iccv13,
url = {http://hal.inria.fr/hal-00873544},
title = {{Multi-View Object Segmentation in Space and Time}},
author = {Djelouah, Abdelaziz and Franco, Jean-S{\'e}bastien and Boyer, Edmond and Le Clerc, Francois and P{\'e}rez, Patrick},
affiliation = {MORPHEO - INRIA Grenoble Rh{\^o}ne-Alpes / LJK Laboratoire Jean Kuntzmann , Technicolor R \& I , Technicolor [Cesson S{\'e}vign{\'e}]},
booktitle = {{ICCV'13 - International Conference On Computer Vision}},
year = {2013},
month = Dec,
pdf = {http://www.cv-foundation.org/openaccess/content_iccv_2013/papers/Djelouah_Multi-view_Object_Segmentation_2013_ICCV_paper.pdf},
}

Comments are closed