Reconstruction of strip-like missing data in geostationary satellite remote sensing imagery based on convolutional neural networks

HE Qi, SHEN Hao, HAO Zengzhou, LI Yunzhou, HUANG Haiqing

Journal of Marine Sciences ›› 2025, Vol. 43 ›› Issue (2) : 39-46.

PDF(4866 KB)
PDF(4866 KB)
Journal of Marine Sciences ›› 2025, Vol. 43 ›› Issue (2) : 39-46. DOI: 10.3969/j.issn.1001-909X.2025.02.005

Reconstruction of strip-like missing data in geostationary satellite remote sensing imagery based on convolutional neural networks

Author information +
History +

Abstract

During satellite operation, sensor malfunctions can lead to irregular strip-like missing areas in imagery, which compromises the integrity of observed information. To address this issue in geostationary satellite remote sensing imagery, a reconstruction model based on convolutional neural networks (CNN)is proposed. The model’s performance is evaluated under different combinations of temporal input data to identify the optimal configuration of consecutive temporal auxiliary data. In the auxiliary data combination, (taking the generation time of the image to be restored as the current time t), when the model takes the previous four phases (t-4, t-3, t-2, t-1)and the previous and subsequent three phases (t-3, t-2, t-1, t+1, t+2, t+3)as inputs respectively, the restoration effect is excellent and can be used for data restoration and reconstruction in real-time and delayed scenarios respectively. Compared with models that use only a single time point as auxiliary data, the proposed model utilizing multi-temporal inputs demonstrates better reconstruction results and higher accuracy. The model is also applicable to the restoration of missing information in other geostationary satellite images.

Key words

geostationary satellite / image inpainting / deep learning / multi-temporal phases / auxiliary data

Cite this article

Download Citations
HE Qi , SHEN Hao , HAO Zengzhou , et al . Reconstruction of strip-like missing data in geostationary satellite remote sensing imagery based on convolutional neural networks[J]. Journal of Marine Sciences. 2025, 43(2): 39-46 https://doi.org/10.3969/j.issn.1001-909X.2025.02.005

References

[1]
杨航, 王素娟, 刘铭坤, 等. FY-3C/VIRR西北太平洋区域海表温度精度评估[J]. 中国海洋大学学报, 2020, 50(12): 151-159.
YANG H, WANG S J, LIU M K, et al. Evaluation of sea surface temperature from FY-3C/VIRR observations in the Northwest Pacific[J]. Periodical of Ocean University of China, 2020, 50(12): 151-159.
[2]
宋晚郊, 张鹏, 孙凌, 等. 基于DINEOF 的风云极轨气象卫星海表温度重构方法研究[J]. 海洋学研究, 2022, 40(2): 10-18.
SONG W J, ZHANG P, SUN L, et al. Reconstruction of sea surface temperature from DINEOF-based FY polar-orbiting meteorological satellite[J]. Journal of Marine Sciences, 2022, 40(2): 10-18.
<span id="__kindeditor_bookmark_start_14__"></span>Sea surface temperature (SST)is the critical factor for depicting the marine thermal distribution. Daily global SST data sets support the typhoon elaborated monitoring and other marine disasters forecast. SST products retrieved by the visible infrared radiometers and mediumresolution imagers have high spatial resolution, while the SST products retrieved by infrared remote sensing are affected by clouds, fog and haze, and therefore a large areas under the clouds are lack of value. SST products retrieved by the microwave radiometer have low spatial resolution, while the microwave could penetrate the cloud layer to achieve all-weather sea surface observation. The data interpolation empirical orthogonal function method (DINEOF)was used to reconstructed the global SST products, and FY-3 (Fengyun 3)SST data sets were applied in this study, which included the SST data sets from the FY-3B/FY-3C Visible and Infra-Red Radiometer, FY-3D Medium Resolution Spectral Imager and FY-3D Micro-Wave Radiation Imager. Accuracy of the reconstructed data sets was verified using OISST measurements to demonstrate the validity and reliability of the DINEOF method. The results show that DINEOF reconstructed sea surface temperature (DSST)data are validated reliable. Root mean square error of the original data is ranging from 0.59 ℃ to 0.70 ℃, while the reconstructed data is relatively stable, ranging from 0.10 ℃ to 0.34 ℃. Correlation coefficient obvious raises from 0.33-0.48 to 0.78-0.98. Multi-sensors reconstructed SST products is continuous and credible in spatial distribution and monitor the variation of warm pool from spring to winter. Addition of FY-3D microwave SST products has significantly improved the spatial continuous distribution and temporal resolution of reconstructed SST.<br>
[3]
李宁, 季辰, 程亮, 等. 遥感影像缺失重建方法对比与时间序列合成[J]. 现代测绘, 2019, 42(5):22-27.
LI N, JI C, CHENG L, et al. The methods comparison of remote sensing image missing reconstruction and time series image synthesis[J]. Modern Surveying and Mapping, 2019, 42(5): 22-27.
[4]
薛兴盛, 吴艳兰. 面向风云静止卫星地表温度产品的缺失数据修复方法对比[J]. 安徽农业大学学报, 2017, 44(2):308-315.
XUE X S, WU Y L. A comparison of missing data reconstruction methods for Feng Yun geostationary satellite land surface temperature products[J]. Journal of Anhui Agricultural University, 2017, 44(2): 308-315.
[5]
张鹏, 刘勇. MOD09A1数据产品中缺失条带的插补方法[J]. 遥感技术与应用, 2015, 30(2):331-336.
Abstract
MODIS地表反射率产品(MOD09A1)是MODIS系列化数据产品中一项重要而基础性的产品。在实际应用中发现该数据产品中仍然继承了原始数据固有的条带缺失问题,且随着数据投影转换,缺失数据在表现为条带状的同时,表现出新的分布特点,以往的插补方法不再适用。利用MOD09A1数据集中描述MODIS数据获取和处理质量的QC数据逐一确定单个缺失像元的准确位置,采用其8邻域内的非缺失像元均值对缺失像元进行插补,在对非条带信息不产生影响的前提下实现了MOD09A1缺失条带的去除,从而确保了该产品数据的质量。选用不同年份不同天数的3景数据进行处理,并将模拟的条带缺失数据采用本文方法处理的结果与真实数据比较,以及将同一条带缺失数据采用不同方法处理的结果比较,结果显示本文方法对于MOD09A1数据条带去除优于以往的方法,并具有普适性和可靠性。
ZHANG P, LIU Y. An interpolation method for stripe missing in the product MOD09A1[J]. Remote Sensing Technology and Application, 2015, 30(2): 331-336.
[6]
陈彦军, 张玉红. 无参考图条件下的Landsat-7 SLC-off图像缺失像素填充算法研究[J]. 地理科学, 2021, 41(7):1276-1284.
Abstract
针对2003年以后Landsat-7扫描线纠正器失效问题,提出一种利用回归分析和三次样条插值对图像缺失像素进行恢复的算法,该方法无需使用参考图,并以非监督分类结果为准则。首先对图像进行预处理,包括条带定位、非监督分类和局部平均灰度计算;然后,利用回归分析计算在每个条带像素的3个方向上条带外侧像素的灰度变化趋势;在非监督分类准则下,通过判断各个方向条带两侧像素类型异同的策略确定条带当前像素的填充方向;随后,对条带边界点和非边界点采用不同算法分别填充。对非边界像素,在填充方向上利用三次样条插值计算当前像素的灰度插值;最后,对条带进行自适应滤波处理。实验结果表明,在不需要参考其它遥感影像与数据的条件下,研究方法与常用的有参考图方法相比较差别不大,而其均方根误差值17.590 4接近于最优的权重线性回归方法的17.400 6。
CHEN Y J, ZHANG Y H. Filling algorithm for missing pixels in Landsat-7 SLC-off images using no reference images[J]. Scientia Geographica Sinica, 2021, 41(7): 1276-1284.

The Landsat-7 scan line corrector failed since 2003, to recover missing pixels in the image we propose a no-reference-image algorithm using regression analysis and cubic spline interpolation under unsupervised classification criteria in this paper. First, in the preprocess stage, stripe location, unsupervised classification and local average gray calculation are carried out. Then, regression analysis is applied to calculate the gray slopes of the pixels in the three directions outside the stripes. Under the unsupervised classification criteria, we determine the filling direction of the current pixel of the stripe by judging the similarities and differences of the types of non-missing pixels on both sides of the strip in each direction. Subsequently, different algorithms are used to fill in the strip boundary points and non-boundary points, respectively. For non-boundary pixels, cubic spline interpolation is performed in the filling direction. Finally, an adaptive filter denoises the stripe area. The experimental results show that no difference can be found from the proposed no-reference-image method to the commonly used methods with reference images. The RMSE value of the proposed method (17.5904)is close to the value of the optimal WLR method (17.4006). The visual effect of the proposed method is better, and notrace of stripes can be found. This algorithm can effectively recover the Landsat-7 satellite SLC-off images after 2003, and provide effective information for further geospatial analysis.

[7]
WANG L L, QU J J, XIONG X X, et al. A new method for retrieving band 6 of Aqua MODIS[J]. IEEE Geoscience and Remote Sensing Letters, 2006, 3(2): 267-270.
[8]
SHEN H F, ZENG C, ZHANG L P. Recovering reflectance of AQUA MODIS band 6 based on within-class local fitting[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2011, 4(1): 185-192.
[9]
CHEN J, ZHU X L, VOGELMANN J E, et al. A simple and effective method for filling gaps in Landsat ETM+ SLC-off images[J]. Remote Sensing of Environment, 2011, 115(4): 1053-1064.
[10]
ZENG C, SHEN H F, ZHANG L P. Recovering missing pixels for Landsat ETM+ SLC-off imagery using multi-temporal regression analysis and a regularization method[J]. Remote Sensing of Environment, 2013, 131: 182-194.
[11]
吕建峰, 邵立珍, 雷雪梅. 基于深度神经网络的图像修复算法综述[J]. 计算机工程与应用, 2023, 59(20):1-12.
Abstract
深度学习的快速发展使计算机视觉技术应用越来越广泛,同时利用深度神经网络根据破损图像的已知信息对图像复原的修复技术成为关注的热点。对近年基于深度神经网络的图像修复方法进行了综述和分析:按照模型优化的方向,对图像修复方法进行分类综述;介绍了图像修复常用的数据集和性能评价指标,并在相关数据集上对各种基于深度神经网络的破损图像修复算法进行性能评价和分析;总结和分析了现有图像修复方法面临的挑战和未来研究方向。
J F, SHAO L Z, LEI X M. Image inpainting algorithm based on deep neural networks[J]. Computer Engineering and Applications, 2023, 59(20): 1-12.
With the rapid development of deep learning, computer vision technology is applied more and more widely. At the same time, the image inpainting technology based on the known information of the damaged image using deep neural network has also become a hot topic. The image inpainting methods based on depth neural network in recent years are reviewed and analyzed. Firstly, the image inpainting methods are classified and summarized according to the view of model optimization. Then the common datasets and performance evaluation indicators are introduced, and the performance evaluation and analysis of various deep neural network-based image inpainting algorithms are carried out on the relevant data sets. Finally, the challenges faced by the existing image inpainting methods are analyzed, and the future research works are prospected.
[12]
彭进业, 余喆, 屈书毅, 等. 基于深度学习的图像修复方法研究综述[J]. 西北大学学报:自然科学版, 2023, 53(6):943-963.
PENG J Y, YU Z, QU S Y, et al. Asurvey of image inpainting methods based on deep learning[J]. Journal of Northwest University: Natural Science Edition, 2023, 53(6): 943-963.
[13]
ZHANG Q, YUAN Q Q, ZENG C, et al. Missing data reconstruction in remote sensing image with a unified spatial-temporal-spectral deep convolutional neural network[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(8): 4274-4288.
[14]
ZHANG Q, YUAN Q Q, LI J, et al. Thick cloud and cloud shadow removal in multitemporal imagery using progressively spatio-temporal patch group deep learning[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 162: 148-160.
[15]
STUCKER C, GARNOT V S F, SCHINDLER K. U-TILISE: A sequence-to-sequence model for cloud removal in optical satellite time series[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 5408716.
[16]
GUO S, YAN Z F, ZHANG K, et al. Toward convolutional blind denoising of real photographs[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, CA, USA, 2019: 1712-1722.
[17]
HU J, SHEN L, SUN G. Squeeze-and-Excitation networks[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA, 2018: 7132-7141.
PDF(4866 KB)

Accesses

Citation

Detail

Sections
Recommended

/