research paper on image restoration

Subscribe to the PwC Newsletter

Join the community, add a new evaluation result row, image restoration.

476 papers with code • 1 benchmarks • 12 datasets

Image Restoration is a family of inverse problems for obtaining a high quality image from a corrupted input image. Corruption may occur due to the image-capture process (e.g., noise, lens blur), post-processing (e.g., JPEG compression), or photography in non-ideal conditions (e.g., haze, motion blur).

Source: Blind Image Restoration without Prior Knowledge

Benchmarks Add a Result

Most implemented papers

Noise2noise: learning image restoration without clean data.

We apply basic statistical reasoning to signal reconstruction by machine learning -- learning to map corrupted observations to clean signals -- with a simple and powerful conclusion: it is possible to learn to restore images by only looking at corrupted examples, at performance at and sometimes exceeding training using clean data, without explicit image priors or likelihood models of the corruption.

Image Restoration Using Convolutional Auto-encoders with Symmetric Skip Connections

In this work, we propose a very deep fully convolutional auto-encoder network for image restoration, which is a encoding-decoding framework with symmetric convolutional-deconvolutional layers.

Deep Image Prior

In this paper, we show that, on the contrary, the structure of a generator network is sufficient to capture a great deal of low-level image statistics prior to any learning.

Learning Enriched Features for Real Image Restoration and Enhancement

With the goal of recovering high-quality image content from its degraded version, image restoration enjoys numerous applications, such as in surveillance, computational photography, medical imaging, and remote sensing.

Restormer: Efficient Transformer for High-Resolution Image Restoration

Since convolutional neural networks (CNNs) perform well at learning generalizable image priors from large-scale data, these models have been extensively applied to image restoration and related tasks.

SwinIR: Image Restoration Using Swin Transformer

In particular, the deep feature extraction module is composed of several residual Swin Transformer blocks (RSTB), each of which has several Swin Transformer layers together with a residual connection.

Simple Baselines for Image Restoration

Although there have been significant advances in the field of image restoration recently, the system complexity of the state-of-the-art (SOTA) methods is increasing as well, which may hinder the convenient analysis and comparison of methods.

The 2018 PIRM Challenge on Perceptual Image Super-resolution

This paper reports on the 2018 PIRM challenge on perceptual super-resolution (SR), held in conjunction with the Perceptual Image Restoration and Manipulation (PIRM) workshop at ECCV 2018.

EnlightenGAN: Deep Light Enhancement without Paired Supervision

yueruchen/EnlightenGAN • 17 Jun 2019

Deep learning-based methods have achieved remarkable success in image restoration and enhancement, but are they still competitive when there is a lack of paired training data?

CycleISP: Real Image Restoration via Improved Data Synthesis

This is mainly because the AWGN is not adequate for modeling the real camera noise which is signal-dependent and heavily transformed by the camera imaging pipeline.

GAN-Based Image Restoration and Colorization

• Conference paper
• First Online: 07 January 2024
• Cite this conference paper

• Aliyah Kabeer 39 ,
• Manali Tanna 39 ,
• K. N. Milinda 39 ,
• Mohammed Uzair Rizwan 39 &
• Pooja Agarwal 39  

Part of the book series: Lecture Notes in Electrical Engineering ((LNEE,volume 1105))

Included in the following conference series:

• International Conference on Emerging Research in Computing, Information, Communication and Applications

105 Accesses

The importance of images in today’s society has made it essential for them to be of the highest quality and visually indicative of their essential traits and attributes. Significant research has been done individually on colorizing and restoring degraded images. Separate studies of Generative Adversarial Networks (GANs) have also been conducted in each of these fields. However, it’s rare to find GAN architectures that can focus on both the tasks at once. With an emphasis on nature photographs, this study proposes a unique GAN architecture that was trained on a customized image dataset including images of landscapes, flowers, and mountains combined with the GoPro Light dataset. The proposed methodology makes use of a combination of different loss functions that enable the model to focus on both tasks simultaneously. Alongside the L1 loss and adversarial loss traditionally used in GANs, the proposed model includes the perceptual loss that performs feature-wise comparisons between images to restore its inherent features. To prove that the GAN can perform both restoration and colorization, its performance has been compared with other models that perform each of the two tasks separately. The model is tested on the curated dataset and evaluated on image-specific metrics like peak signal-to-noise ratio (PSNR) and structural similarity Index (SSIM). The model gives results that compare well with existing models, and it can colorize and restore images that have been degraded with motion blur or camera misfocus—successfully striking a good balance between the two tasks. The paper concludes by providing insight into the future work that can be carried out.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Durable hardcover edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Arnaud R (2020) Landscape pictures, version 1. https://www.kaggle.com/datasets/arnaud58/landscape-pictures . Accessed 2022

Belitskaya O (2017) The dataset of flower images, version 7. https://www.kaggle.com/code/olgabelitskaya/the-dataset-of-flower-images/data . Accessed 2022

Gampala V, Sunil Kumar M, Sushama C, Fantin Irudaya Raj E (2020) Deep learning based image processing approaches for image deblurring. Mater Today Proc. https://doi.org/10.1016/j.matpr.2020.11.076

Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2020) Generative adversarial networks. Commun ACM 63(11):139–144. https://doi.org/10.1145/3422622

He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of IEEE conference computer vision pattern recognition (CVPR), pp 770–778

Google Scholar  

Intel image classification challenge, mountain class. https://www.kaggle.com/puneet6060/intel-image-classification

Ledig C, Theis L, Huszar F, Caballero J, Cunning-ham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z, Twitter WS (2017) Photo-realistic single image super-resolution using a generative adversarial network

Mourchid Y, Donias M, Berthoumieu Y (2021) Automatic image colorization based on multi-discriminators generative adversarial networks. In: 2020 28th European signal processing conference (EUSIPCO), pp 1532–1536. https://doi.org/10.23919/Eusipco47968.2020.9287792

Nazeri K, Ng E, Ebrahimi M (2018) Image colorization using generative adversarial networks. In: Perales F, Kittler J (eds) Articulated motion and deformable objects. AMDO 2018. Lecture notes in computer science, vol 10945. Springer, Cham. https://doi.org/10.1007/978-3-319-94544-6 9

Nguyen T, Mori K, Thawonmas R (2016) Image colorization using a deep convolutional neural network. arXiv preprint arXiv:1604.07904

Pan X, Zhan X, Dai B, Lin D, Loy CC, Luo P (2021) Exploiting deep generative prior for versatile image restoration and manipulation. In: IEEE transactions on pattern analysis and machine intelligence

Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, Cham

Sankar R, Nair A, Abhinav P, Mothukuri SKP, Koolagudi SG (2020) Image colorization using GANs and perceptual loss. In: 2020 international conference on artificial intelligence and signal processing (AISP), pp 1–4. https://doi.org/10.1109/AISP48273.2020.9073284

Sarmai A, Sadhu A (2022) Image colorization using convolutional autoencoders and generative adversarial networks, vol 9. IARJSET, pp 86–99. https://doi.org/10.17148/IARJSET.2022.9115

Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: ICLR

Sophia SGG, Umar S, Visvas N (2022) An efficient method for blind image restoration using GAN. In: 2022 international conference on innovative computing, intelligent communication and smart electrical systems (ICSES), pp 1–8. https://doi.org/10.1109/ICSES55317.2022.9914234

Thomas BS, Dogra R, Dixit B, Raut A (2018) Automatic image and video colourisation using deep learning. In: 2018 international conference on smart city and emerging technology (ICSCET), pp 1–4. https://doi.org/10.1109/IC-SCET.2018.8537308

Tomosada H, Kudo T, Fujisawa T, Ikehara M (2021) GAN-based image deblurring using DCT loss with customized datasets. IEEE Access 9:135224–135233. https://doi.org/10.1109/ACCESS.2021.3116194

Article   Google Scholar  

Wang X, Yu K, Wu S, Gu J, Liu Y, Dong C, Loy CC, Qiao Y, Tang X (2018) ESRGAN: enhanced super-resolution generative adversarial networks. ECCV Workshops

Wang X, Li Y, Zhang H, Shan Y (2021) Towards real-world blind face restoration with generative facial prior. In: 2021 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 9164–9174. https://doi.org/10.1109/CVPR46437.2021.00905

Xiao Y, Jiang A, Liu C, Wang M (2019) Single image colorization via modified cyclegan. In: 2019 IEEE international conference on image processing (ICIP), pp 3247–3251. https://doi.org/10.1109/ICIP.2019.8803677

Zhao JLL, Bai H, Zeng AWB, Zhao Y (2019) Simultaneous colordepth super-resolution with conditional generative adversarial networks. arXiv:1708.09105

Zhu J-Y, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE international conference on computer vision, pp 2223–2232

Download references

Author information

Authors and affiliations.

Department of Computer Science and Engineering, PES University, Bangalore, India

Aliyah Kabeer, Manali Tanna, K. N. Milinda, Mohammed Uzair Rizwan & Pooja Agarwal

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Aliyah Kabeer .

Editor information

Editors and affiliations.

Central University of Karnataka, Kalaburagi, Karnataka, India

N. R. Shetty

Nitte Meenakshi Institute of Technology, Bengaluru, Karnataka, India

N. H. Prasad

H. C. Nagaraj

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Cite this paper.

Kabeer, A., Tanna, M., Milinda, K.N., Rizwan, M.U., Agarwal, P. (2024). GAN-Based Image Restoration and Colorization. In: Shetty, N.R., Prasad, N.H., Nagaraj, H.C. (eds) Advances in Communication and Applications . ERCICA 2023. Lecture Notes in Electrical Engineering, vol 1105. Springer, Singapore. https://doi.org/10.1007/978-981-99-7633-1_40

Download citation

DOI : https://doi.org/10.1007/978-981-99-7633-1_40

Published : 07 January 2024

Publisher Name : Springer, Singapore

Print ISBN : 978-981-99-7632-4

Online ISBN : 978-981-99-7633-1

eBook Packages : Engineering Engineering (R0)

Share this paper

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

•  We're Hiring!
•  Help Center

Image Restoration

• Most Cited Papers
• Most Downloaded Papers
• Newest Papers
• Save to Library
• Last »
• Presidential rhetoric Follow Following
• Image Deblurring and Restoration Follow Following
• Mood Disorders Follow Following
• Public Deliberation Follow Following
• Political Rhetoric Follow Following
• Participatory Democracy Follow Following
• Political Discourse Analysis Follow Following
• Rhetorical Criticism Follow Following
• Depressive Disorder Follow Following
• Affective Disorders Follow Following

Enter the email address you signed up with and we'll email you a reset link.

• Academia.edu Publishing
•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

Help | Advanced Search

Computer Science > Computer Vision and Pattern Recognition

Title: diffusion models for image restoration and enhancement -- a comprehensive survey.

Abstract: Image restoration (IR) has been an indispensable and challenging task in the low-level vision field, which strives to improve the subjective quality of images distorted by various forms of degradation. Recently, the diffusion model has achieved significant advancements in the visual generation of AIGC, thereby raising an intuitive question, "whether diffusion model can boost image restoration". To answer this, some pioneering studies attempt to integrate diffusion models into the image restoration task, resulting in superior performances than previous GAN-based methods. Despite that, a comprehensive and enlightening survey on diffusion model-based image restoration remains scarce. In this paper, we are the first to present a comprehensive review of recent diffusion model-based methods on image restoration, encompassing the learning paradigm, conditional strategy, framework design, modeling strategy, and evaluation. Concretely, we first introduce the background of the diffusion model briefly and then present two prevalent workflows that exploit diffusion models in image restoration. Subsequently, we classify and emphasize the innovative designs using diffusion models for both IR and blind/real-world IR, intending to inspire future development. To evaluate existing methods thoroughly, we summarize the commonly-used dataset, implementation details, and evaluation metrics. Additionally, we present the objective comparison for open-sourced methods across three tasks, including image super-resolution, deblurring, and inpainting. Ultimately, informed by the limitations in existing works, we propose five potential and challenging directions for the future research of diffusion model-based IR, including sampling efficiency, model compression, distortion simulation and estimation, distortion invariant learning, and framework design.

Submission history

Access paper:.

• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

Supervise-Assisted Self-Supervised Deep-Learning Method for Hyperspectral Image Restoration

Ieee account.

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

MIT Technology Review

• Newsletters

Google helped make an exquisitely detailed map of a tiny piece of the human brain

A small brain sample was sliced into 5,000 pieces, and machine learning helped stitch it back together.

• Cassandra Willyard archive page

A team led by scientists from Harvard and Google has created a 3D, nanoscale-resolution map of a single cubic millimeter of the human brain. Although the map covers just a fraction of the organ—a whole brain is a million times larger—that piece contains roughly 57,000 cells, about 230 millimeters of blood vessels, and nearly 150 million synapses. It is currently the highest-resolution picture of the human brain ever created.

To make a map this finely detailed, the team had to cut the tissue sample into 5,000 slices and scan them with a high-speed electron microscope. Then they used a machine-learning model to help electronically stitch the slices back together and label the features. The raw data set alone took up 1.4 petabytes. “It’s probably the most computer-intensive work in all of neuroscience,” says Michael Hawrylycz, a computational neuroscientist at the Allen Institute for Brain Science, who was not involved in the research. “There is a Herculean amount of work involved.”

Many other brain atlases exist, but most provide much lower-resolution data. At the nanoscale, researchers can trace the brain’s wiring one neuron at a time to the synapses, the places where they connect. “To really understand how the human brain works, how it processes information, how it stores memories, we will ultimately need a map that’s at that resolution,” says Viren Jain, a senior research scientist at Google and coauthor on the paper, published in Science on May 9 . The data set itself and a preprint version of this paper were released in 2021 .

Brain atlases come in many forms. Some reveal how the cells are organized. Others cover gene expression. This one focuses on connections between cells, a field called “connectomics.” The outermost layer of the brain contains roughly 16 billion neurons that link up with each other to form trillions of connections. A single neuron might receive information from hundreds or even thousands of other neurons and send information to a similar number. That makes tracing these connections an exceedingly complex task, even in just a small piece of the brain..  

To create this map, the team faced a number of hurdles. The first problem was finding a sample of brain tissue. The brain deteriorates quickly after death, so cadaver tissue doesn’t work. Instead, the team used a piece of tissue removed from a woman with epilepsy during brain surgery that was meant to help control her seizures.

Once the researchers had the sample, they had to carefully preserve it in resin so that it could be cut into slices, each about a thousandth the thickness of a human hair. Then they imaged the sections using a high-speed electron microscope designed specifically for this project. 

Next came the computational challenge. “You have all of these wires traversing everywhere in three dimensions, making all kinds of different connections,” Jain says. The team at Google used a machine-learning model to stitch the slices back together, align each one with the next, color-code the wiring, and find the connections. This is harder than it might seem. “If you make a single mistake, then all of the connections attached to that wire are now incorrect,” Jain says. 

“The ability to get this deep a reconstruction of any human brain sample is an important advance,” says Seth Ament, a neuroscientist at the University of Maryland. The map is “the closest to the  ground truth that we can get right now.” But he also cautions that it’s a single brain specimen taken from a single individual. 

The map, which is freely available at a web platform called Neuroglancer , is meant to be a resource other researchers can use to make their own discoveries. “Now anybody who’s interested in studying the human cortex in this level of detail can go into the data themselves. They can proofread certain structures to make sure everything is correct, and then publish their own findings,” Jain says. (The preprint has already been cited at least 136 times .) 

The team has already identified some surprises. For example, some of the long tendrils that carry signals from one neuron to the next formed “whorls,” spots where they twirled around themselves. Axons typically form a single synapse to transmit information to the next cell. The team identified single axons that formed repeated connections—in some cases, 50 separate synapses. Why that might be isn’t yet clear, but the strong bonds could help facilitate very quick or strong reactions to certain stimuli, Jain says. “It’s a very simple finding about the organization of the human cortex,” he says. But “we didn’t know this before because we didn’t have maps at this resolution.”

The data set was full of surprises, says Jeff Lichtman, a neuroscientist at Harvard University who helped lead the research. “There were just so many things in it that were incompatible with what you would read in a textbook.” The researchers may not have explanations for what they’re seeing, but they have plenty of new questions: “That’s the way science moves forward.” 

Biotechnology and health

How scientists traced a mysterious covid case back to six toilets.

When wastewater surveillance turns into a hunt for a single infected individual, the ethics get tricky.

An AI-driven “factory of drugs” claims to have hit a big milestone

Insilico is part of a wave of companies betting on AI as the "next amazing revolution" in biology

• Antonio Regalado archive page

The quest to legitimize longevity medicine

Longevity clinics offer a mix of services that largely cater to the wealthy. Now there’s a push to establish their work as a credible medical field.

• Jessica Hamzelou archive page

There is a new most expensive drug in the world. Price tag: $4.25 million

But will the latest gene therapy suffer the curse of the costliest drug?

Stay connected

Get the latest updates from mit technology review.

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at [email protected] with a list of newsletters you’d like to receive.