TY - CONF
T1 - Adaptive lighting for data-driven non-line-of-sight 3D localization and object identification
AU - Chandran, Sreenithy
AU - Jayasuriya, Suren
N1 - Funding Information:
Acknowledgments: We would like to thank Srinivasa Narasimhan, Kyros Kutulakos, and Ioannis Gkioulekas for initial discussions. Shenbagaraj Kannapiran from ASU helped render illustrations used in this paper. The AME Fabrication lab also assisted with 3D printing, scene setup, and lending the projector for our hardware prototype. This work was partially supported by the Defense Advanced Research Projects Agency (REVEAL Grant HR00111620021) to S. Jayasuriya, as well as joint support from both the Herberger Research Initiative in the Herberger Institute for Design and the Arts (HIDA) and the Fulton Schools of Engineering (FSE) at ASU to S. Chandran and S. Jayasuriya. This work was also supported by a hardware GPU donation by NVIDIA.
Publisher Copyright:
© 2019. The copyright of this document resides with its authors.
PY - 2020
Y1 - 2020
N2 - Non-line-of-sight (NLOS) imaging of objects not visible to either the camera or illumination source is a challenging task with vital applications including surveillance and robotics. Recent NLOS reconstruction advances have been achieved using time-resolved measurements which requires expensive and specialized detectors and laser sources. In contrast, we propose a data-driven approach for NLOS 3D localization and object identification requiring only a conventional camera and projector. To generalize to complex line-of-sight (LOS) scenes with non-planar surfaces and occlusions, we introduce an adaptive lighting algorithm. This algorithm, based on radiosity, identifies and illuminates scene patches in the LOS which most contribute to the NLOS light paths, and can factor in system power constraints. We achieve an average identification of 87.1% object identification for four classes of objects, and average localization of the NLOS object's centroid with a mean-squared error (MSE) of 1.97 cm in the occluded region for real data taken from a hardware prototype. These results demonstrate the advantage of combining the physics of light transport with active illumination for data-driven NLOS imaging.
AB - Non-line-of-sight (NLOS) imaging of objects not visible to either the camera or illumination source is a challenging task with vital applications including surveillance and robotics. Recent NLOS reconstruction advances have been achieved using time-resolved measurements which requires expensive and specialized detectors and laser sources. In contrast, we propose a data-driven approach for NLOS 3D localization and object identification requiring only a conventional camera and projector. To generalize to complex line-of-sight (LOS) scenes with non-planar surfaces and occlusions, we introduce an adaptive lighting algorithm. This algorithm, based on radiosity, identifies and illuminates scene patches in the LOS which most contribute to the NLOS light paths, and can factor in system power constraints. We achieve an average identification of 87.1% object identification for four classes of objects, and average localization of the NLOS object's centroid with a mean-squared error (MSE) of 1.97 cm in the occluded region for real data taken from a hardware prototype. These results demonstrate the advantage of combining the physics of light transport with active illumination for data-driven NLOS imaging.
UR - http://www.scopus.com/inward/record.url?scp=85087340536&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85087340536&partnerID=8YFLogxK
M3 - Paper
AN - SCOPUS:85087340536
T2 - 30th British Machine Vision Conference, BMVC 2019
Y2 - 9 September 2019 through 12 September 2019
ER -