TY - JOUR
T1 - Optical flow for video super-resolution
T2 - a survey
AU - Tu, Zhigang
AU - Li, Hongyan
AU - Xie, Wei
AU - Liu, Yuanzhong
AU - Zhang, Shifu
AU - Li, Baoxin
AU - Yuan, Junsong
N1 - Publisher Copyright:
© 2022, The Author(s), under exclusive licence to Springer Nature B.V.
PY - 2022/12
Y1 - 2022/12
N2 - Video super-resolution is currently one of the most active research topics in computer vision as it plays an important role in many visual applications. Generally, video super-resolution contains a significant component, i.e., motion compensation, which is used to estimate the displacement between successive video frames for temporal alignment. Optical flow, which can supply dense and sub-pixel motion between consecutive frames, is among the most common ways for this task. To obtain a good understanding of the effect that optical flow acts in video super-resolution, in this work, we conduct a comprehensive review on this subject for the first time. This investigation covers the following major topics: the function of super-resolution (i.e., why we require super-resolution); the concept of video super-resolution (i.e., what is video super-resolution); the description of evaluation metrics (i.e., how (video) super-resolution performs); the introduction of optical flow based video super-resolution; the investigation of using optical flow to capture temporal dependency for video super-resolution. Prominently, we give an in-depth study of the deep learning based video super-resolution method, where some representative algorithms are analyzed and compared. Additionally, we highlight some promising research directions and open issues that should be further addressed.
AB - Video super-resolution is currently one of the most active research topics in computer vision as it plays an important role in many visual applications. Generally, video super-resolution contains a significant component, i.e., motion compensation, which is used to estimate the displacement between successive video frames for temporal alignment. Optical flow, which can supply dense and sub-pixel motion between consecutive frames, is among the most common ways for this task. To obtain a good understanding of the effect that optical flow acts in video super-resolution, in this work, we conduct a comprehensive review on this subject for the first time. This investigation covers the following major topics: the function of super-resolution (i.e., why we require super-resolution); the concept of video super-resolution (i.e., what is video super-resolution); the description of evaluation metrics (i.e., how (video) super-resolution performs); the introduction of optical flow based video super-resolution; the investigation of using optical flow to capture temporal dependency for video super-resolution. Prominently, we give an in-depth study of the deep learning based video super-resolution method, where some representative algorithms are analyzed and compared. Additionally, we highlight some promising research directions and open issues that should be further addressed.
KW - Optical Flow-based video super-resolution
KW - Optical flow
KW - Temporal dependency
KW - Video super-resolution
UR - http://www.scopus.com/inward/record.url?scp=85134534198&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85134534198&partnerID=8YFLogxK
U2 - 10.1007/s10462-022-10159-8
DO - 10.1007/s10462-022-10159-8
M3 - Article
AN - SCOPUS:85134534198
SN - 0269-2821
VL - 55
SP - 6505
EP - 6546
JO - Artificial Intelligence Review
JF - Artificial Intelligence Review
IS - 8
ER -