TY - JOUR
T1 - Turbulence strength C2 n estimation from video using physics-based deep learning
AU - Saha, Ripon Kumar
AU - Salcin, Esen
AU - Kim, Jihoo
AU - Smith, Joseph
AU - Jayasuriya, Suren
N1 - Funding Information:
Acknowledgments. This work was supported by the Army SBIR Phase II Contract No. W91CRB21C0028, and NSF IIS-1909192. The authors wish to thank Cameron Whyte for helpful discussions at the start of this project.
Publisher Copyright:
© 2022 Authors. All rights reserved.
PY - 2022/10/24
Y1 - 2022/10/24
N2 - Images captured from a long distance suffer from dynamic image distortion due to turbulent flowof air cells with random temperatures, and thus refractive indices. This phenomenon, known as image dancing, is commonly characterized by its refractive-index structure constant C2 n as a measure of the turbulence strength. For many applications such as atmospheric forecast model, long-range/astronomy imaging, and aviation safety, optical communication technology, C2 n estimation is critical for accurately sensing the turbulent environment. Previous methods for C2 n estimation include estimation from meteorological data (temperature, relative humidity, wind shear, etc.) for single-point measurements, two-ended pathlength measurements from optical scintillometer for path-averaged C2 n, and more recently estimating C2 n from passive video cameras for low cost and hardware complexity. In this paper, we present a comparative analysis of classical image gradient methods for C2 n estimation and modern deep learning-based methods leveraging convolutional neural networks. To enable this, we collect a dataset of video capture along with reference scintillometer measurements for ground truth, and we release this unique dataset to the scientific community. We observe that deep learning methods can achieve higher accuracy when trained on similar data, but suffer from generalization errors to other, unseen imagery as compared to classical methods. To overcome this trade-off, we present a novel physics-based network architecture that combines learned convolutional layers with a differentiable image gradient method that maintains high accuracy while being generalizable across image datasets.
AB - Images captured from a long distance suffer from dynamic image distortion due to turbulent flowof air cells with random temperatures, and thus refractive indices. This phenomenon, known as image dancing, is commonly characterized by its refractive-index structure constant C2 n as a measure of the turbulence strength. For many applications such as atmospheric forecast model, long-range/astronomy imaging, and aviation safety, optical communication technology, C2 n estimation is critical for accurately sensing the turbulent environment. Previous methods for C2 n estimation include estimation from meteorological data (temperature, relative humidity, wind shear, etc.) for single-point measurements, two-ended pathlength measurements from optical scintillometer for path-averaged C2 n, and more recently estimating C2 n from passive video cameras for low cost and hardware complexity. In this paper, we present a comparative analysis of classical image gradient methods for C2 n estimation and modern deep learning-based methods leveraging convolutional neural networks. To enable this, we collect a dataset of video capture along with reference scintillometer measurements for ground truth, and we release this unique dataset to the scientific community. We observe that deep learning methods can achieve higher accuracy when trained on similar data, but suffer from generalization errors to other, unseen imagery as compared to classical methods. To overcome this trade-off, we present a novel physics-based network architecture that combines learned convolutional layers with a differentiable image gradient method that maintains high accuracy while being generalizable across image datasets.
UR - http://www.scopus.com/inward/record.url?scp=85140862082&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85140862082&partnerID=8YFLogxK
U2 - 10.1364/OE.469976
DO - 10.1364/OE.469976
M3 - Article
C2 - 36299011
AN - SCOPUS:85140862082
SN - 1094-4087
VL - 30
SP - 40854
EP - 40870
JO - Optics Express
JF - Optics Express
IS - 22
ER -