Super-Resolution Imaging

Author

Patrick Vandewalle

Abstract

Aliasing in images is often considered as a nuisance. Artificial low frequency patterns and jagged edges appear when an image is sampled at a too low frequency. However, aliasing also conveys useful information about the high frequency content of the image, which is exploited in super-resolution applications. We use a set of input images of the same scene to extract such high frequency information and create a higher resolution aliasing-free image. Typically, there is a small shift or more complex motion between the different images, such that they contain slightly different information about the scene.

super-resolution imaging super-resolution imaging
Figure 1: High resolution, aliasing-free images are reconstructed from sets of low resolution, aliased images. Images are aligned and reconstructed using subspace methods.

Super-resolution image reconstruction can be formulated as a multichannel sampling problem with unknown offsets. This results in a set of equations that are linear in the unknown signal coefficients but nonlinear in the offsets. We concentrate on the computation of these offsets, as they are an essential prerequisite for an accurate high resolution reconstruction. If a part of the image spectra is free of aliasing, the planar shift and rotation parameters can be computed using only this low frequency information. In such a case, the images can be registered pairwise to a reference image. Such a method is not applicable if the images are undersampled by a factor of two or larger. A higher number of images needs to be registered jointly. Two subspace methods are developed for such highly aliased images. The first approach is based on a Fourier description of the aliased signals as a sum of overlapping parts of the spectrum. It uses a rank condition to find the correct offsets. The second one uses a more general expansion in an arbitrary Hilbert space to compute the signal offsets. The sampled signal is represented as a linear combination of sampled basis functions. The offsets are computed by projecting the signal onto varying subspaces.

Under certain conditions, in particular for bandlimited signals, the nonlinear super-resolution equations can be written as a set of polynomial equations. Using Buchberger’s algorithm, the solution can then be computed as a Gröbner basis for the corresponding polynomial ideal. After a description of a standard algorithm, adaptations are made for the use with noisy measurements.

We tested our algorithms in simulations and practical experiments. The experiments are performed on sets of real images taken with a digital camera. The results show the validity of the algorithms: registration parameters are computed with subpixel precision, and aliasing is accurately removed from the resulting high resolution image.

Collaborations

Luciano Sbaiz
Sabine Süsstrunk
Martin Vetterli

Publications

P. Vandewalle, L. Sbaiz, J. Vandewalle and M. Vetterli, Aliasing is Good for You: Joint Registration and Reconstruction for Super-Resolution, Technical Report, 2006.
[detailed record] [bibtex]

P. Vandewalle, S. Susstrunk and M. Vetterli, Superresolution images reconstructed from aliased images, Proc. SPIE/IS&T Visual Communications and Image Processing Conference, Vol. 5150, pp. 1398-1405, 2003.
[detailed record] [bibtex]

P. Vandewalle, S. Susstrunk and M. Vetterli, Double resolution from a set of aliased images, Proc. IS&T/SPIE Electronic Imaging 2004: Sensors and Camera Systems for Scientific, Industrial, and Digital Photography Applications V, Vol. 5301, pp. 374-382, 2004.
[detailed record] [bibtex]

P. Vandewalle, L. Sbaiz, J. Vandewalle and M. Vetterli, How to take advantage of aliasing in bandlimited signals, IEEE Conference on Acoustics, Speech and Signal Processing, Vol. 3, pp. 948-951, 2004.
[detailed record] [bibtex]

P. Vandewalle, L. Sbaiz, M. Vetterli and S. Susstrunk, Super-Resolution from Highly Undersampled Images, Proc. IEEE International Conference on Image Processing, Vol. 1, pp. 889-892, 2005.
[detailed record] [bibtex]

P. Vandewalle, L. Sbaiz, S. Susstrunk and M. Vetterli, Registration of Aliased Images for Super-Resolution Imaging, Proc. SPIE/IS&T Visual Communications and Image Processing Conference, Vol. 6077, pp. 13-23, 2006.
[detailed record] [bibtex]

P. Vandewalle, L. Sbaiz and M. Vetterli, Signal Reconstruction from Multiple Unregistered Sets of Samples using Groebner Bases, Proc. IEEE Conference on Acoustics, Speech and Signal Processing, Vol. 3, pp. 604-607, 2006.
[detailed record] [bibtex]

P. Vandewalle, K. Krichane, D. Alleysson and S. Susstrunk, Joint Demosaicing and Super-Resolution Imaging from a Set of Unregistered Aliased Images, Proc. IS&T/SPIE Electronic Imaging: Digital Photography III, Vol. 6502, 2007.
[detailed record] [bibtex]

A. Ajdari Rad, L. Meylan, P. Vandewalle and S. Susstrunk, Multidimensional image enhancement from a set of unregistered differently exposed images, Proc. IS&T/SPIE Electronic Imaging: Computational Imaging V, Vol. 6498, 2007.
[detailed record] [bibtex]

P. Vandewalle, S. Susstrunk and M. Vetterli, A Frequency Domain Approach to Registration of Aliased Images with Application to Super-Resolution, EURASIP Journal on Applied Signal Processing (special issue on Super-resolution), Vol. 2006, pp. Article ID 71459, 14 pages, 2006.
[detailed record] [bibtex]

P. Vandewalle, L. Sbaiz, J. Vandewalle and M. Vetterli, Super-Resolution from Unregistered and Totally Aliased Signals Using Subspace Methods, IEEE Transactions on Signal Processing, Vol. 55, Nr. 7, Part 2, pp. 3687-3703, 2007.
[detailed record] [bibtex]

L. Sbaiz, P. Vandewalle and M. Vetterli, Groebner Basis Methods for Multichannel Sampling with Unknown Offsets, Applied and Computational Harmonic Analysis, Vol. 25, Nr. 3, pp. 277-294, 2008.
[detailed record] [bibtex]

P. Vandewalle, Super-resolution from unregistered aliased images, Ph.D. Thesis, 2006.
[detailed record] [bibtex]

Funding

National Competence Center in Research on Mobile Information and Communication Systems (NCCR-MICS), a center supported by the Swiss National Science Foundation under grant number 5005-67322.