Progressive Update Guided Interdependent Networks for Single Image Dehazing

Aupendu Kar, Sobhan Kanti Dhara, Debashis Sen, Prabir Kumar Biswas

Department of Electronics and Electrical Communication Engineering
Indian Institute of Technology Kharagpur, India

Use the slider to compare before and after

City after removing the smog or fog using iterative dehazing technique

Metro Rail after removing the smog or fog using iterative dehazing technique

Railway Bridge after removing the smog or fog using iterative dehazing technique

Abstract

Images with haze of different varieties often pose a significant challenge to dehazing. Therefore, guidance by estimates of haze parameters related to the variety would be beneficial, and their progressive update jointly with haze reduction will allow effective dehazing. To this end, we propose a multi-network dehazing framework containing novel interdependent dehazing and haze parameter updater networks that operate in a progressive manner. The haze parameters, transmission map and atmospheric light, are first estimated using dedicated convolutional networks that allow color-cast handling. The estimated parameters are then used to guide our dehazing module, where the estimates are progressively updated by novel convolutional networks. The updating takes place jointly with progressive dehazing using a network that invokes inter-step dependencies. The joint progressive updating and dehazing gradually modify the haze parameter values toward achieving effective dehazing. Through different studies, our dehazing framework is shown to be more effective than image-to-image mapping and predefined haze formation model based dehazing. The framework is also found capable of handling a wide variety of hazy conditions wtih different types and amounts of haze and color casts. Our dehazing framework is qualitatively and quantitatively found to outperform the state-of-the-art on synthetic and real-world hazy images of multiple datasets with varied haze conditions.

Schematic Framework

Training Pipeline for iterative dehazing technique — Schematic framework of our proposed dehazing approach

Highlights

We propose an end-to-end dehazing module that progressively dehazes hazy images using interdependent dehazing and updater networks.
We introduce novel haze parameter updater networks that update initial estimates of transmission map and atmospheric light to guide the dehazing process.
We propose a dehazing network that performs the refined dehazing jointly with the progressive updating by the updaters while invoking inter-step dependencies.
To handle color cast in hazy images, channel-wise atmospheric light is initially estimated using a novel deep network and then updated in the dehazing module.

Model Architecture: Iterative Dehazing

Algorithm for Iterative Transmission and Atmospheric Light Update based Recurrent
Neural Network for Single Image Dehazing — Proposed Dehazing Algorithm

Architectural Model for Iterative Transmission and Atmospheric Light Update based Recurrent
Neural Network for Single Image Dehazing — Our Proposed LSTM based Iterative Dehazing Network

Model Architecture: Transmission and Atmospheric

Ambient Model for Iterative Transmission and Atmospheric Light Update based Recurrent
Neural Network for Single Image Dehazing — Proposed Atmospheric Light Estimation Model

Transmission Model for Iterative Transmission and Atmospheric Light Update based Recurrent
Neural Network for Single Image Dehazing — Densely Connected Transmission Map Estimation Model [1]

Comparison with other Models

Image collage for a set of Hazy images. — Hazy

Filtered Hazy images after using FSID — FSID [1]

Filtered Hazy images after using MSBDN model. — MSBDN [2]

Filtered Hazy images after using Fast Single Image Dehazing model. — Refine [3]

Filtered Hazy images after using D4. — D4 [4]

Filtered Hazy images after using Iterative Transmission and Atmospheric Light Update based Recurrent
Neural Network for Single Image Dehazing — Proposed PUG-D

Ablation Studies

Hazy image of a market place — Hazy Image

Using ResNet on the Hazy image. — ResNet16

Using LResX2 on the Hazy image. — ResNet6+A+T

Using LResXLSTM2 on the Hazy image. — ResNet6+A+T+LSTM

Using LResXC2 on the Hazy image. — ResNet6+A+T+IUN

Using LResXLSTMC2 on the Hazy image. — ResNet6+A+T+IUN+LSTM

Download

Code

Google drive image icon — Training & Testing Datasets

References

[1] Kim, Se Eun, Tae Hee Park, and Il Kyu Eom. "Fast single image dehazing using saturation based transmission map estimation." IEEE Transactions on Image Processing 29 (2019): 1985-1998.
[2] Dong, H., Pan, J., Xiang, L., Hu, Z., Zhang, X., Wang, F., & Yang, M. H. (2020). Multi-scale boosted dehazing network with dense feature fusion. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 2157-2167).
[3] Zhao, S., Zhang, L., Shen, Y., & Zhou, Y. (2021). RefineDNet: A weakly supervised refinement framework for single image dehazing. IEEE Transactions on Image Processing, 30, 3391-3404.
[4] Yang, Y., Wang, C., Liu, R., Zhang, L., Guo, X., & Tao, D. (2022). Self-Augmented Unpaired Image Dehazing via Density and Depth Decomposition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 2037-2046).
[5] Ju, M., Ding, C., Ren, W., & Yang, Y. (2022). IDBP: Image dehazing using blended priors including non-local, local, and global priors. IEEE Transactions on Circuits and Systems for Video Technology.
[6] Zhang, X., Wang, J., Wang, T., & Jiang, R. (2022). Hierarchical feature fusion with mixed convolution attention for single image dehazing. IEEE Transactions on Circuits and Systems for Video Technology, 32(2), 510-522.
[6] Mandal, M., Meedimale, Y.R., Reddy, M.S.K. and Vipparthi, S.K., 2022. Neural Architecture Search for Image Dehazing. IEEE Transactions on Artificial Intelligence.

Citation (BibTeX)

@misc{kar2022progressive,
title={Progressive Update Guided Interdependent Networks for Single Image Dehazing},
author={Aupendu Kar and Sobhan Kanti Dhara and Debashis Sen and Prabir Kumar Biswas},
year={2023},
eprint={2008.01701},
archivePrefix={arXiv},
primaryClass={cs.CV}
}