Joint Semantic and Motion Segmentation for Dynamic Scenes using Deep Convolutional Networks

Nazrul Haque, Dinesh Reddy, K. Madhava Krishna

2017

Abstract

Dynamic scene understanding is a challenging problem and motion segmentation plays a crucial role in solving it. Incorporating semantics and motion enhances the overall perception of the dynamic scene. For applications of outdoor robotic navigation, joint learning methods have not been extensively used for extracting spatiotemporal features or adding different priors into the formulation. The task becomes even more challenging without stereo information being incorporated. This paper proposes an approach to fuse semantic features and motion clues using CNNs, to address the problem of monocular semantic motion segmentation. We deduce semantic and motion labels by integrating optical flow as a constraint with semantic features into dilated convolution network. The pipeline consists of three main stages i.e Feature extraction, Feature amplification and Multi Scale Context Aggregation to fuse the semantics and flow features. Our joint formulation shows significant improvements in monocular motion segmentation over the state of the art methods on challenging KITTI tracking dataset.

Paper Citation

in Harvard Style

Haque N., Reddy D. and Madhava Krishna K. (2017). Joint Semantic and Motion Segmentation for Dynamic Scenes using Deep Convolutional Networks . In Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 5: VISAPP, (VISIGRAPP 2017) ISBN 978-989-758-226-4, pages 75-85. DOI: 10.5220/0006129200750085

in Bibtex Style

@conference{visapp17,
author={Nazrul Haque and Dinesh Reddy and K. Madhava Krishna},
title={Joint Semantic and Motion Segmentation for Dynamic Scenes using Deep Convolutional Networks},
booktitle={Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 5: VISAPP, (VISIGRAPP 2017)},
year={2017},
pages={75-85},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006129200750085},
isbn={978-989-758-226-4},
}

in EndNote Style

TY - CONF
JO - Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 5: VISAPP, (VISIGRAPP 2017)
TI - Joint Semantic and Motion Segmentation for Dynamic Scenes using Deep Convolutional Networks
SN - 978-989-758-226-4
AU - Haque N.
AU - Reddy D.
AU - Madhava Krishna K.
PY - 2017
SP - 75
EP - 85
DO - 10.5220/0006129200750085