A two-stage restoration of distorted underwater images using compressive sensing and image registration

doi:10.1007/s40436-020-00340-z

Advances in Manufacturing ›› 2021, Vol. 9 ›› Issue (2): 273-285.doi: 10.1007/s40436-020-00340-z

A two-stage restoration of distorted underwater images using compressive sensing and image registration

Zhen Zhang, Yu-Gui Tang, Kuo Yang

School of Mechatronic Engineering and Automation, Shanghai University, Shanghai 200444, People's Republic of China

收稿日期:2020-07-21 修回日期:2020-10-31 出版日期:2021-06-25 发布日期:2021-05-24
通讯作者: Zhen Zhang E-mail:zhangzhen_ta@shu.edu.cn
基金资助:
This work was supported by the National Key R&D Program of China (Grant No. 2018YFB1309200).

A two-stage restoration of distorted underwater images using compressive sensing and image registration

Zhen Zhang, Yu-Gui Tang, Kuo Yang

School of Mechatronic Engineering and Automation, Shanghai University, Shanghai 200444, People's Republic of China

Received:2020-07-21 Revised:2020-10-31 Online:2021-06-25 Published:2021-05-24
Contact: Zhen Zhang E-mail:zhangzhen_ta@shu.edu.cn
Supported by:
This work was supported by the National Key R&D Program of China (Grant No. 2018YFB1309200).

摘要/Abstract

摘要： Imaging through a time-varying water surface exhibits severe non-rigid geometric distortions and motion blur. Theoretically, although the water surface possesses smoothness and temporal periodicity, random fluctuations are inevitable in an actual video sequence. Meanwhile, considering the distribution of information, the image structure contributes more to the restoration. In this paper, a new two-stage restoration method for distorted underwater video sequences is presented. During the first stage, salient feature points, which are selected through multiple methods, are tracked across the frames, and the motion fields at all pixels are estimated using a compressive sensing solver to remove the periodic distortions. During the second stage, the combination of a guided filter algorithm and an image registration method is applied to remove the structural-information-oriented residual distortions. Finally, the experiment results show that the method outperforms other state-of-the-art approaches in terms of the recovery effect and time.

The full text can be downloaded at https://link.springer.com/article/10.1007/s40436-020-00340-z

关键词: Periodic wave, Random wave, Compressive sensing (CS), Image registration (IR), Structural information

Abstract: Imaging through a time-varying water surface exhibits severe non-rigid geometric distortions and motion blur. Theoretically, although the water surface possesses smoothness and temporal periodicity, random fluctuations are inevitable in an actual video sequence. Meanwhile, considering the distribution of information, the image structure contributes more to the restoration. In this paper, a new two-stage restoration method for distorted underwater video sequences is presented. During the first stage, salient feature points, which are selected through multiple methods, are tracked across the frames, and the motion fields at all pixels are estimated using a compressive sensing solver to remove the periodic distortions. During the second stage, the combination of a guided filter algorithm and an image registration method is applied to remove the structural-information-oriented residual distortions. Finally, the experiment results show that the method outperforms other state-of-the-art approaches in terms of the recovery effect and time.

The full text can be downloaded at https://link.springer.com/article/10.1007/s40436-020-00340-z

Key words: Periodic wave, Random wave, Compressive sensing (CS), Image registration (IR), Structural information

Zhen Zhang, Yu-Gui Tang, Kuo Yang. A two-stage restoration of distorted underwater images using compressive sensing and image registration[J]. Advances in Manufacturing, 2021, 9(2): 273-285.

参考文献

1. Gilles J, Osher S (2016) Wavelet burst accumulation for turbulence mitigation. J Electron Imaging 25(3):033003
2. Panetta K, Gao C, Agaian S (2015) Human-visual-system-inspired underwater image quality measures. IEEE J Ocean Eng 41(3):541-551
3. Paul N, De Chillaz A, Collette J (2015) On-line restoration for turbulence degraded video in nuclear power plant reactors. Signal Image Video Process 9(3):601-610
4. Turlaev DG, Dolin LS (2013) On observing underwater objects through a wavy water surface:a new algorithm for image correction and laboratory experiment. Izvestiya Atmos Ocean Phys 49(3):339-345
5. Zhang Z, Yang X (2019) Bio-inspired motion planning for reaching movement of a manipulator based on intrinsic tau jerk guidance. Adv Manuf 7:315-325
6. Holohan ML, Dainty JC (1997) Low-order adaptive optics:a possible use in underwater imaging? Opt Laser Technol 29(1):51-55
7. Murase H (1992) Surface shape reconstruction of a nonrigid transparent object using refraction and motion. IEEE Trans Pattern Anal Mach Intell 14:1045-1052
8. Seemakurthy K, Rajagopalan AN (2015) Deskewing of underwater images. IEEE Trans Image Process 24(3):1046-1059
9. Tian Y, Narasimhan SG (2009) Seeing through water:image restoration using model-based tracking. In:Proceedings of the IEEE 12th international conference on computer vision. IEEE, Kyoto, pp 2303-2310
10. Cox C, Munk W (1956) Slopes of the sea surface deduced from photographs of sun glitter. Bull Scripps Inst Oceanogr 6:401-479
11. Efros A, Isler V, Shi J et al (2004) Seeing through water. In:Proceedings of conference and workshop on neural information processing systems. Neural Information Processing Systems Foundation, Vancouver, pp 393-400
12. Donate A, Dahme G, Ribeiro E (2006) Classification of textures distorted by water waves. In:Proceedings of international conference on pattern recognition. IEEE, Hong Kong, pp 421-424
13. Donate A, Ribeiro E (2006) Improved reconstruction of images distorted by water waves. In:Advances in computer graphics and computer vision. Springer, Berlin, pp 264-277
14. Wen Z, Lambert A, Fraser D et al (2010) Bispectral analysis and recovery of images distorted by a moving water surface. Appl Opt 49(33):6376-6384
15. Kanaev AV, Ackerman J, Fleet E et al (2009) Imaging through the air-water interface. In:Proceedings of computational optical sensing and imaging. OSA, San Jose, pp 13-15
16. Kanaev AV, Hou W, Restaino SR et al (2015) Restoration of images degraded by underwater turbulence using structure tensor oriented image quality (STOIQ) metric. Opt Express 23(13):17077-17090
17. Kanaev AV, Hou W, Restaino SR et al (2014) Correction methods for underwater turbulence degraded imaging. In:Proceedings of remote sensing of clouds and the atmosphere XIX; and optics in atmosphereic propagation and adaptive systems XVII. Proc. SPIE, 92421P
18. Zhang R, He D, Li Y et al (2018) Synthetic imaging through wavy water surface with centroid evolution. Opt Express 26(20):26009-26019
19. Lau CP, Lai YH, Lui LM (2019) Variational models for joint subsampling and reconstruction of turbulence-degraded images. J Sci Comput 78(3):1488-1525
20. Lau CP, Lai YH, Lui LM (2019) Reconstruction of atmospheric turbulence-distorted images via RPCA and quasiconformal maps. Inverse Probl 35(7):074002
21. Oreifej O, Guang S, Pace T et al (2011) A two-stage reconstruction approach for seeing through water. In:Proceedings of IEEE conference on computer vision and pattern recognition. IEEE, Colorado, pp 1153-1160
22. Halder KK, Paul M, Tahtali M et al (2017) Correction of geometrically distorted underwater images using shift map analysis. J Opt Soc Am A 34(4):666-673
23. Zhang Z, Yang X (2019) Reconstruction of distorted underwater images using robust registration. Opt Express 27(7):9996-10008
24. Li Z, Murez Z, Kriegman D et al (2018) Learning to see through turbulent water. In:Proceedings of 2018 IEEE winter conference on applications of computer vision. IEEE, Lake Tahoe, pp 512-520
25. Lau CP, Yung CP, Lui LM (2018) Image retargeting via beltrami representation. IEEE Trans Image Process 27(12):5787-5801
26. James JG, Agrawal P, Rajwade A (2019) Restoration of nonrigidly distorted underwater image using a combination of compressive sensing and local polynomial image representations. In:Proceedings of the IEEE international conference on computer vision. IEEE, Seoul, pp 7839-7848
27. He K, Sun J, Tang X (2012) Guided image filtering. IEEE Trans Pattern Anal Mach Intell 35(6):1397-1409
28. Bay H, Tuytelaars T, Van GL (2006) Surf:speeded up robust features. In:Proceedings of European conference on computer vision. Graz, pp 404-417
29. Rosten E, Porter R, Drummond T (2008) Faster and better:a machine learning approach to corner detection. IEEE Trans Pattern Anal Mach Intell 32(1):105-119
30. Leutenegger S, Chli M, Siegwart RY (2011) BRISK:Binary robust invariant scalable keypoints. In:Proceedings of international conference on computer vision. IEEE, Barcelona, pp 2548-2555
31. Nam T, Kim S, Jung D (2019) Hardware implementation of KLT tracker for real-time intruder dectection and tracking using onboard camera. Int J Aeronaut Space Sci 20(1):300-314
32. Nakanishi-Ohno Y, Hukushima K (2018) Data-driven diagnosis for compressed sensing with cross validation. Phys Rev E 98(5):052120
33. Sun T, Tang YG, Zhang Z (2019) Structural information reconstruction of distorted underwater images using image registration. Appl Sci 10(16):5670
34. Rueckert D, Sonoda LI, Hayes C et al (1999) Nonrigid registration using freeform deformations:application to breast MR images. IEEE Trans Med Imaging 18(8):712-721
35. Berdinsky D, Kim TW, Cho D et al (2015) Based of T-meshes and the refinement of hierarchical B-splines. Comput Methods Appl Mech Eng 283:841-855
36. Xue W, Zhang L, Mou XQ et al (2013) Gradient magnitude similarity deviation:a highly efficient perceptual image quality index. IEEE Trans Image Process 23(2):684-695
37. Zhang L, Zhang L, Mou XQ et al (2011) FSIM:a feature similarity index for image quality assessment. IEEE Trans Image Process 20(8):2378-2386

A two-stage restoration of distorted underwater images using compressive sensing and image registration

A two-stage restoration of distorted underwater images using compressive sensing and image registration

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 0

编辑推荐

Metrics

本文评价