1.

Introduction

Digital images are often corrupted by artifacts due to noise in the imaging system, digitization, and compression. Smoothing is a widely used technique to obtain more visually pleasing images. Several methods have been proposed in the literature to reduce edge blurring when smoothing is applied. Among the edge sharpening techniques, the unsharp masking approach is widely used to improve the perceptual quality of an image. This is a linear filtering, and although it is easy to implement, it also enhances noises, digitization effects, and blocking artifacts. This often results in visually unpleasing images. Several methods have also been proposed in the literature that focus on smoothing and preserving the details in digital images. In particular, nonlinear techniques have been extensively used. Polynomial filters are frequently used to replace the high-pass filter in the unsharp masking scheme.^{1, 2} In Ref. 3, the proposed structure is similar to the conventional unsharp masking, except that the enhancement is allowed only in the direction of the maximal change, and the enhancement parameter is computed as a nonlinear function of the rate of change. A modification of the Laplacian, which is called the order statistic (OS) Laplacian, is considered in Ref. 4. A variable length nonlinear filter that has the capability to both sharpen the edges and smooth out the noise is presented in Ref. 5. In Ref. 6, an adaptive algorithm is introduced so that a sharpening action is performed only in locations where the image exhibits significant dynamics. An adaptive directional filtering is also performed to provide suitable emphasis on the different directional characteristics of the details. However, the solutions cited may still introduce some artifacts due to the amplification of the input disturbances, in particular within the flat areas. They often require weighting parameters, which have to be set on the basis of human intuition or judgment.

In this paper, we present a modified version of the unsharp masking scheme that is able to enhance image quality. This method performs image smoothing, not only preserving but also enhancing the salient details in images. Our algorithm is based on the consideration that there is a strong relationship between biological vision and image rendering. In particular, the image rendering process will be more successful in interpreting the original scene and in applying the appropriate transformations. The key idea is to locally process the image according to topographic maps obtained from a neurodynamical model of visual attention. This overcomes the trade-off between smoothing and sharpening, typical of the traditional approaches.

2.

Neurodynamical Model of Visual Attention

In the last decades, several computational attention methodologies have been studied, and they have become a powerful tool in vision systems. Computational neuroscience provides a mathematical framework to study the mechanisms involved in brain functions, such as visual attention mechanisms. It is well known that only part of the visual information of a given scene is processed in full detail, while the remainder is left relatively unprocessed. Therefore, visual attention facilitates the processing of that limited portion of the input associated with the relevant information, and it suppresses the remaining information. The first attention model was proposed by Koch and Ullman.⁷ Itti and Koch,⁸ then defined a visual attention system based on saliency maps to predict visually salient features of a scene. Chauvin ⁹ and Guironnet ¹⁰ proposed models inspired by the retina and the primary visual cortex cell functionalities. Corchs and Deco¹¹ implemented a neurodynamical model for visual attention, based on evidence from functional, neurophysiological, and psychological findings.¹² This visual model consists of interconnected modules that can be related to certain components of the visual cortex. In the application of this paper, we have adopted a reduced version of this model, which is the bottom-up component, given by the module that represents the primary visual cortex. Given an input image, this information enters the visual cortex through this module, named V1. From this theoretical model, we can calculate the neural activities of the V1 neurons, which correspond to the internal representation of the image. This representation is the map of activities of the given input image (Fig. 1). The mathematical formulation and a detailed explanation of the neurodynamical model can be found in Refs. 12, 13.

Fig. 1

(a) Original image; (b) corresponding topographic map of the salient regions obtained by the neurodynamical model of visual attention.

3.

Low-Quality Image Enhancement Algorithm

In our algorithm, the enhanced image $f_e(x,y)$ is obtained correcting the original $f(x,y)$ by subtracting nonsalient high frequencies $f_{hNS}(x,y)$ , producing a smoothing effect, and by adding salient high frequencies, $f_{hS}(x,y)$ , with a consequent edge sharpening:

4.

Experimental Results

We have tested our algorithm on a data set composed of 300 color images of different sizes (from $120\times 160$ to $2272\times 1704$ pixels), resolutions, and quality. These images are mostly landscapes, portraits of people, and indoor groups of people and were downloaded from personal Web pages or acquired using various digital cameras and scanners. The quality of these images in terms of JPEG compression, noise, dynamic range, and so on varied and was in most cases unknown. The edge enhancement algorithm works only on the intensity channel to avoid possible color distortion due to saturation. We have compared our algorithm with the traditional unsharp masking applied to the original image previously smoothed by a Gaussian filter. The sigma of this filter varies according to the image size, and it is chosen to produce a smoothing equivalent to the one obtained by applying our method. The enhanced image $f_e(x,y)$ is obtained adding a fraction $\lambda$ of its high-pass filtered version $f_h(x,y)$ to the smoothed image $f(x,y)$ as follows:

The constant factor $\lambda$ has the same value, $\lambda =0.2$ , in Eq. 1 and in Eq. 2, for a better comparison. Even though unsharp masking is simple and produces good results in many applications, its main drawback is that it does not distinguish between significant and nonsignificant high frequencies, such as noise, and thus all these high frequencies are added with the same weight $\lambda$ . As a result, the traditional algorithm applied to the original low-quality image also enhances noise, digitization effects, and blocking artifacts. A previous Gaussian smoothing mitigates the unpleasing artifacts of the low-quality image, but it also blurs the edges, as shown in Fig. 2b. Our algorithm [Fig. 2c], by exploiting topographic maps of visually salient regions, instead produces an output image more pleasing to the observer, while still permitting an adequate edge enhancement. In fact, only high frequencies corresponding to regions that are nonsignificant to our visual system are smoothed, while significant details are sharpened. In Fig. 3, the high frequencies of Fig. 1, which are added by applying the traditional unsharp masking, are compared to the salient frequencies added with our method. A simple experimental comparison has been carried out by a panel of five experts. In 81% of the cases, our method has been judged by the majority of the experts as better than the Gaussian smoothed version of the unsharp masking [Eq. 2], which was preferred by the experts in only 10% of the cases. In the remaining 9% of the cases, the two methods have been judged equivalent. The enhanced images were always judged better than the not-processed original images. More examples are available at www.ivl.disco.unimib.it/Activities/Edge%20sharpening.htm

Fig. 2

(a) The considered portion of the original image in Fig. 1. (b) The same portion, blurred by a Gaussian smoothing and then enhanced with the traditional unsharp masking. (c) The same portion after applying our method.

Fig. 3

(a) All the high frequencies of the image in Fig. 1. (b) The corresponding salient high frequencies.

5.

Conclusions

This paper presents a modified version of the unsharp masking method that makes it possible to reduce the sharpening sensitivity to noise and also due to digitization effects and compression. Moreover, being based on a neurodynamical model of human visual attention, the processed images appear more natural and pleasing to observers. The preliminary experimental results on a data set of about 300 images of various subjects with various types of degradations demonstrate that our method has good performance with respect to the traditional unsharp masking combined with smoothing. We plan to investigate further the relationship among user preferences, image content, image defects and artifacts, and our image enhancement based on the Corchs-Deco visual attention model. To this end, a completely new data set is under construction in which the images are degraded in a completely controlled manner. Such a study will also include the influence of the adopted visual attention model in the proposed approach. Different visual attention models generate maps that may be visually different. Our preliminary study indicates the suitability of the Corchs-Deco model because it identifies regions that present strong edges and that correspond to high contrast areas. However, the model is somewhat expensive from the computational point of view. Other models such as Ref. 8 should be implemented therefore and extensively compared with ours in terms of both effectiveness and efficiency. A comparison with more sophisticated enhancement methods (e.g., Refs. 5, 6) would also be interesting.