TY - GEN
T1 - Robust lensless image reconstruction via PSF Estimation
AU - Rego, Joshua D.
AU - Kulkarni, Karthik
AU - Jayasuriya, Suren
N1 - Funding Information:
We wish to thank Celine Cheung and Yasser Dbeis for help with lensless camera simulation and video dataset creation respectively. This work was supported by NSF grant IIS-1909192, a seed grant from ASU s Herberger Research Initiative, as well as GPU resources from ASU Research Computing.
Funding Information:
Acknowledgements: We wish to thank Celine Cheung and Yasser Dbeis for help with lensless camera simulation and video dataset creation respectively. This work was supported by NSF grant IIS-1909192, a seed grant from ASU’s Herberger Research Initiative, as well as GPU resources from ASU Research Computing.
Publisher Copyright:
© 2021 IEEE.
PY - 2021/1
Y1 - 2021/1
N2 - Lensless imaging is a new, emerging modality where image sensors utilize optical elements in front of the sensor to perform multiplexed imaging. There have been several recent papers to reconstruct images from lensless imagers, including methods that utilize deep learning for state-of-the-art performance. However, many of these methods require explicit knowledge of the optical element, such as the point spread function, or learn the reconstruction mapping for a single fixed PSF. In this paper, we explore a neural network architecture that performs joint image reconstruction and PSF estimation to robustly recover images captured with multiple PSFs from different cameras. Using adversarial learning, this approach achieves improved reconstruction results that do not require explicit knowledge of the PSF at test-time and shows an added improvement in the reconstruction model's ability to generalize to variations in the camera's PSF. This allows lensless cameras to be utilized in a wider range of applications that require multiple cameras without the need to explicitly train a separate model for each new camera.
AB - Lensless imaging is a new, emerging modality where image sensors utilize optical elements in front of the sensor to perform multiplexed imaging. There have been several recent papers to reconstruct images from lensless imagers, including methods that utilize deep learning for state-of-the-art performance. However, many of these methods require explicit knowledge of the optical element, such as the point spread function, or learn the reconstruction mapping for a single fixed PSF. In this paper, we explore a neural network architecture that performs joint image reconstruction and PSF estimation to robustly recover images captured with multiple PSFs from different cameras. Using adversarial learning, this approach achieves improved reconstruction results that do not require explicit knowledge of the PSF at test-time and shows an added improvement in the reconstruction model's ability to generalize to variations in the camera's PSF. This allows lensless cameras to be utilized in a wider range of applications that require multiple cameras without the need to explicitly train a separate model for each new camera.
UR - http://www.scopus.com/inward/record.url?scp=85113505895&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85113505895&partnerID=8YFLogxK
U2 - 10.1109/WACV48630.2021.00045
DO - 10.1109/WACV48630.2021.00045
M3 - Conference contribution
AN - SCOPUS:85113505895
T3 - Proceedings - 2021 IEEE Winter Conference on Applications of Computer Vision, WACV 2021
SP - 403
EP - 412
BT - Proceedings - 2021 IEEE Winter Conference on Applications of Computer Vision, WACV 2021
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2021 IEEE Winter Conference on Applications of Computer Vision, WACV 2021
Y2 - 5 January 2021 through 9 January 2021
ER -