Empirical mode decomposition (EMD) is a simple, local, adaptive, and efficient method for nonlinear and nonstationary signal analysis. However, for dealing with multidimensional signals, EMD and its variants such as bidimensional EMD (BEMD) and multidimensional EMD (MEMD) are very slow due to the needs of a large amount of envelope interpolations. Recently, a method called iterative filtering has been proposed. This filtering-based method is not as precise as EMD but its processing speed is very fast and can achieve comparable results as EMD does in many image and signal processing applications. We combine quaternion algebra and iterative filtering to achieve the edge detection, color quantization, segmentation, texture removal, and noise reduction task of color images. We can obtain similar results by using quaternion combined with EMD; however, as mentioned before, EMD is slow and cumbersome. Therefore, we propose to use quaternion iterative filtering as an alternative method for quaternion EMD (QEMD). The edge of color images can be detected by using intrinsic mode functions (IMFs) and the color quantization results can be obtained from residual image. The noise reduction algorithm of our method can be used to deal with Gaussian, salt-and-pepper, speckle noise, etc. The peak signal-to-noise ratio results are satisfactory and the processing speed is also very fast. Since textures in a color image are high-frequency components, we also can use quaternion iterative filtering to decompose a color image into many high- and low-frequency IMFs and remove textures by eliminating high-frequency IMFs.

An approach is presented to solve the problem of spatial shift wrapping associated with spatial shift estimation-based fringe pattern profilometry (FPP). This problem arises as the result of fringe reuses (that is, use of fringes with periodic light intensity variance), and the spatial shift can only be identified without ambiguity within the range of a fringe width. It is demonstrated that the problem is similar to the phase unwrapping problem associated with the phase-detection-based FPP, and the proposed method is inspired by the existing ideas of using multiple images with different wavelengths proposed for phase unwrapping. The effectiveness of the proposed method is verified by comparing experimental results against several objects, with the last object consisting of more complex surface features. We conclude by showing that our method is successful in reconstructing the fine details of the more complex object.

This paper describes an approach for small infrared (IR) target detection using frequency-spatial cues. We model the background as spikes of the amplitude spectrum in the frequency domain. Target regions are highlighted through background suppression, and the suppression is realized via convoluting the amplitude spectrum with a low-pass Gaussian kernel of an appropriate scale. A theoretical analysis of the convoluting process in the frequency domain is presented. We note that the high values are attributed to sharp gradients in the IR image. In order to uniformly highlight the target region, the proposed algorithm introduces cues of image segmentation in the spatial domain. Targets are completely preserved in the final result. An image database is built, which is used to test the proposed algorithm. Results show that our algorithm detects small IR targets effectively with a competitive performance over some state-of-the-art techniques, even for images with cluttered backgrounds. In addition, we show that it is able to detect multiple targets with varied sizes, which are challenges for existing algorithms.

We present a color-to-gray conversion algorithm that retains both the overall appearance and the discriminability of details of the input color image. The algorithm employs a weighted pyramid image fusion scheme to blend the R, G, and B color channels of the input image into a single grayscale image. The use of simple visual quality metrics as weights in the fusion scheme serves to retain visual contrast from each of the input color channels. We demonstrate the effectiveness of the method by qualitative and quantitative comparison with several state-of-the-art methods.

Large oblique stereo images are of great interest because they have a large coverage and a high reconstruction precision. However, severe distortions are more likely to occur in this type of image. This can make conventional algorithms inaccurate, computationally expensive, and invalid. We present a new robust quasidense image matching algorithm for large oblique images. Our algorithm can be divided into two steps. First, we find a sufficient number of highly accurate seed matches that are uniformly distributed by integrating complementary affine invariant feature matching with least-squares matching. Second, we consider quasidense matches covering overlapping areas of images. We use match propagation beginning with the best seed, iteratively applying the local perspective invariant neighborhood transform (PINT) with the normalized cross-correlation metric. The local PINT is dynamically updated using the current new match set, and the erroneous matches are eliminated according to their geometric consistency. We conducted experiments on simulated and real large oblique images to demonstrate that the proposed algorithm is effective and can robustly find quasidense matches. Comparisons with the existing methods demonstrated that it is superior in terms of accuracy and efficiency.

We propose a multilevel halftone-based color error diffusion (ED) algorithm. Our algorithm aims to achieve a “blue noise” color halftone by exploiting the interchannel relationship of a separated color image. The method focuses on dot distribution and dot overlapping control to yield high quality results. It treats color ED as a multilevel halftone and employs an embedded multilevel ED algorithm to determine the number of color dots in the processing pixel. In the color rendering process, the most suitable colors are selected as output based on the sum of the input and the error diffused from the processed dot. Experimental results show that the proposed algorithm generates high-quality halftone images with “blue noise” characteristics. Comparisons with state-of-the-art methods using objective and subjective measures show that our method is superior, as it effectively avoids dot overlapping and preserves the details of the original image.

In recent years, sparse representation-based classification (SRC) has received significant attention due to its high recognition rate. However, the original SRC method requires a rigid alignment, which is crucial for its application. Therefore, features such as SIFT descriptors are introduced into the SRC method, resulting in an alignment-free method. However, a feature-based dictionary always contains considerable useful information for recognition. We explore the relationship of the similarity of the SIFT descriptors to multitask recognition and propose a clustering-weighted SIFT-based SRC method (CWS-SRC). The proposed approach is considerably more suitable for multitask recognition with sufficient samples. Using two public face databases (AR and Yale face) and a self-built car-model database, the performance of the proposed method is evaluated and compared to that of the SRC, SIFT matching, and MKD-SRC methods. Experimental results indicate that the proposed method exhibits better performance in the alignment-free scenario with sufficient samples.

A visible region of interest (ROI) extraction method based on region expansion is proposed. It is a method that is combined with a new reverse realization of calibration based on homography. The proposed method extracts the ROI automatically and successfully and simultaneously calibrates the distortion caused by the stereoscopic camera system. It is a practical and easy method because it does not require any extra processing before display. It can be used to discard the confusing margin of the synthesized image of several parallax images and reduce the data amount needed to be processed and encoded. Here, real-time capturing, transmitting, and three-dimensional displaying systems are demonstrated to verify its effectiveness and feasibility.

The intention of shape coding in the MPEG-4 is to improve the coding efficiency as well as to facilitate the object-oriented applications, such as shape-based object recognition and retrieval. These require both efficient shape compression and effective shape description. Although these two issues have been intensively investigated in data compression and pattern recognition fields separately, it remains an open problem when both objectives need to be considered together. To achieve high coding gain, the operational rate-distortion optimal framework can be applied, but the direction restriction of the traditional eight-direction edge encoding structure reduces its compression efficiency and description effectiveness. We present two arbitrary direction edge encoding structures to relax this direction restriction. They consist of a sector number, a short component, and a long component, which represent both the direction and the magnitude information of an encoding edge. Experiments on both shape coding and hand gesture recognition validate that our structures can reduce a large number of encoding vertices and save up to 48.9% bits. Besides, the object contours are effectively described and suitable for the object-oriented applications.

The recently published joint demosaicking and zooming algorithms for single-sensor digital cameras all overfit the popular Kodak test images, which have been found to have higher spectral correlation than typical color images. Their performance perhaps significantly degrades on other datasets, such as the McMaster test images, which have weak spectral correlation. A new joint demosaicking and zooming algorithm is proposed for the Bayer color filter array (CFA) pattern, in which the edge direction information (edge map) extracted from the raw CFA data is consistently used in demosaicking and zooming. It also moderately utilizes the spectral correlation between color planes. The experimental results confirm that the proposed algorithm produces an excellent performance on both the Kodak and McMaster datasets in terms of both subjective and objective measures. Our algorithm also has high computational efficiency. It provides a better tradeoff among adaptability, performance, and computational cost compared to the existing algorithms.

We present two improvement techniques for stereo matching algorithms using silicon retina sensors. We verify the results with ground truth data. In contrast to conventional monochrome/color cameras, silicon retina sensors deliver an asynchronous flow of events instead of common framed and discrete intensity or color images. While using this kind of sensor in a stereo setup to enable new fields of applications, it also introduces new challenges in terms of stereo image analysis. Using this type of sensor, stereo matching algorithms have to deal with sparse event data, thus, less information. This affects the quality of the achievable disparity results and renders improving the stereo matching algorithms a necessary task. For this reason, we introduce two techniques for increasing the accuracy of silicon retina stereo results, in the sense that the average distance error is reduced. The first method is an adapted belief propagation approach optimizing the initial matching cost volume, and the second is an innovative two-stage postfilter for smoothing and outlier rejection. The evaluation shows that the proposed techniques increase the accuracy of the stereo matching and constitute a useful extension for using silicon retina sensors for depth estimation.

We propose a method for removing mismatches from the given putative point correspondences in image pairs based on “coherent spatial relations.” Under the Bayesian framework, we formulate our approach as a maximum likelihood problem and solve a coherent spatial relation between the putative point correspondences using an expectation–maximization (EM) algorithm. Our approach associates each point correspondence with a latent variable indicating it as being either an inlier or an outlier, and alternatively estimates the inlier set and recovers the coherent spatial relation. It can handle not only the case of image pairs with rigid motions but also the case of image pairs with nonrigid motions. To parameterize the coherent spatial relation, we choose two-view geometry and thin-plate spline as models for rigid and nonrigid cases, respectively. The mismatches could be successfully removed via the coherent spatial relations after the EM algorithm converges. The quantitative results on various experimental data demonstrate that our method outperforms many state-of-the-art methods, it is not affected by low initial correct match percentages, and is robust to most geometric transformations including a large viewing angle, image rotation, and affine transformation.

A phase-searching look-up table (PS-LUT) method is proposed to dramatically reduce the memory size of the new look-up table (N-LUT) method as well as to keep its advantage of fast computational speed. Small quantities of samples are chosen as the basic phase points (BPPs) in the principle fringe pattern (PFP), and phases of the object points for BPPs are precalculated and stored in the table. The phases of object point for the whole PFP could be quickly obtained through a phase-searching method. With proper reference beam phases, PFP could be rapidly generated. The experimental results reveal that the computational speed of the proposed method is about 24 times faster than that of the ray-tracing method, and the required memory size is 1731 times less than that of the N-LUT method. To eliminate the coherent noise of the N-LUT and PS-LUT, the random PFP N-LUT (RN-LUT) and PS-LUT (RPS-LUT), in which the PFP is randomly chosen for every object point, are put forward. The reconstructed images with a spatial light modulator indicate that both the RN-LUT and RPS-LUT methods are extremely effective in improving the quality of the reconstructed images.

This paper addresses how to define and control the perceived depth when rendering a single stereoscopic three-dimensional (3-D) model. In most 3-D manipulation software, the control of view navigation is the most important feature needed to visualize 3-D scenes of good quality. In a stereoscopic 3-D situation, however, this becomes more complex. We used two factors, parallax range and average parallax, to quantify the 3-D effect of rendering a 3-D model. After an experiment using subjective questionnaires, the fusional limit and depth perception of 22 subjects were regressed as paraboloid functions of parallax range and average parallax. Then, the comfort region, which is defined according to these parameters, was used again for developing an auto-adjustment algorithm for stereoscopic view navigation. This algorithm iteratively adjusts the parameters of a virtual stereo camera and simultaneously restrains parallax range and average parallax within a comfortable region. Finally, by using questionnaires and critical fusion frequency tests, we verified that this algorithm can significantly improve the comfort index of a user in customary operations.

This paper presents an efficient lossless compression algorithm, the coding tree assignment scheme with principal index-pattern coding algorithm (CTAS-PIPCA), to encode image vector quantization (VQ). The coding model is designed on the basis of the schemes proposed in the previous works to further improve the coding performance of coding tree assignment scheme with improved search-order coding algorithm (CTAS-ISOC) by PIPCA. The PIPCA technique exploits the correlation of neighboring index pairs not in the original vector-quantized index map but in the principal index-pattern table which is generated from the two-dimensional histogram of index patterns in the training stage. The CTAS-PIPCA method is evaluated via extensive experiments. The searching matched index in the principal index-pattern table results in lower time complexity than CTAS-ISOC. The results also show that the proposed technique apparently reduces the bit rate as compared to the conventional VQ and other existing popular lossless index coding schemes, such as SOC and CTAS-ISOC.

A global robust M-estimation scheme for maximum a posteriori (MAP) image super-resolution which efficiently addresses the presence of outliers in the low-resolution images is proposed. In iterative MAP image super-resolution, the objective function to be minimized involves the highly resolved image, a parameter controlling the step size of the iterative algorithm, and a parameter weighing the data fidelity term with respect to the smoothness term. Apart from the robust estimation of the high-resolution image, the contribution of the proposed method is twofold: (1) the robust computation of the regularization parameters controlling the relative strength of the prior with respect to the data fidelity term and (2) the robust estimation of the optimal step size in the update of the high-resolution image. Experimental results demonstrate that integrating these estimations into a robust framework leads to significant improvement in the accuracy of the high-resolution image.

A local Fisher discriminant analysis can work well for a multimodal problem. However, it often suffers from the undersampled problem, which makes the local within-class scatter matrix singular. We develop a supervised discriminant analysis technique called uncorrelated regularized local Fisher discriminant analysis for image feature extraction. In this technique, the local within-class scatter matrix is approximated by a full-rank matrix that not only solves the undersampled problem but also eliminates the poor impact of small and zero eigenvalues. Statistically uncorrelated features are obtained to remove redundancy. A trace ratio criterion and the corresponding iterative algorithm are employed to globally solve the objective function. Experimental results on four famous face databases indicate that our proposed method is effective and outperforms the conventional dimensionality reduction methods.

This paper examines three Bayesian statistical segmentation techniques with an innovative attenuation compensation on synthetic data and breast ultrasound medical images. All use expectation maximization for estimating the Gaussian model parameters and segment the data using a three-dimensional (3-D) Markov random field pixel neighborhood. This paper compares three Bayesian segmentation techniques: maximum a posteriori simulated annealing (MAP-SA), MAP iterated conditional modes (MAP-ICM), and maximization of posterior marginals (MPM). We conclude that because of the high speckle noise and adverse attenuation challenges of breast ultrasound, the MPM algorithm has the best performance. This is due to better localized segmentation than the other MAP techniques. We present results first with synthetic images then with breast ultrasound. Our new contributions for a 3-D breast ultrasound produce improved results using a model of the noise, in which the Gaussian mean is proportional to the image attenuation with depth, combined with a new prior probability model.

We propose a method that automatically detects mura defects on display film surfaces using a modified version of Weber’s law. The proposed method detects mura defects regardless of their properties and shapes by identifying regions perceived by human vision as mura using the brightness of pixel and image distribution ratio of mura in an image histogram. The proposed detection method comprises five stages. In the first stage, the display film surface image is acquired and a gray-level shift performed. In the second and third stages, the image histogram is acquired and analyzed, respectively. In the fourth stage, the mura range is acquired. This is followed by postprocessing in the fifth stage. Evaluations of the proposed method conducted using 200 display film mura image samples indicate a maximum detection rate of ∼95.5%. Further, the results of application of the Semu index for luminance mura in flat panel display (FPD) image quality inspection indicate that the proposed method is more reliable than a popular conventional method.

Orthogonal rotation invariant moments (ORIMs) exhibit attractive characteristics such as rotation invariance (translation and scale invariance can be made after normalization process), robustness against noise, good image reconstruction capability, and low information redundancy. Therefore, these are the most commonly used global techniques for image description in different digital image processing applications. The ORIM features are, however, unable to represent local features. Wavelet transforms (WTs), on the other hand, can effectively represent local features. WTs are not invariant to geometric transformation. We propose hybrid face recognition methods based on ORIMs and WTs. In these methods, WTs are used to represent the local features of face images and also to reduce the dimensionality of the face image which results in reducing the large computational time required for moments. WTs retain the characteristics of the original image even after reducing the size of the image. Moreover, the low frequency subband of WTs provides coefficients which are less sensitive to facial expression variation. The role of ORIMs is to capture highly discriminative rotation invariant global features with minimum redundancy. Detail experimental results show that the proposed hybrid methods based on the combination of WTs and ORIMs outperform the ORIMs methods with low processing time.

Viewpoint variation has been a major challenge in dealing with comparison-based image processing. Reduction or total removal of viewpoint variation has been the common pursuit of many processing applications such as human motion analysis and gesture analysis. By exploiting three-dimensional (3-D) skeletal joints information provided by RGB-D cameras such as Kinect, this study proposes a skeleton-based viewpoint invariant transformation (SVIT) technique that transforms the 3-D skeleton data into an orthogonal coordinate system constructed by the three most stable joints of detected person’s upper torso: the left shoulder, the right shoulder, and the spine. The proposed 3-D transformation can eliminate observation variations caused not only by viewpoint differences but also individual differences in body swing movement. With reference to the human activity database MSRDailyAct3D and our recording data, experiments on human motion analysis were designed and conducted to contrast the effectiveness of the proposed SVIT as a preprocessing step before comparing the test data and the sample data, with very different viewpoints. This results show that, with SVIT as a pre-processing for activity analysis, the performance of correct activity identification has increased from 45% to 60%. SVIT outperforms other state-of-the-art view invariant human motion analysis based on human body centric approaches.

Most of the devices for acquisition and display of medical images use rectangular lattices even though there are other sampling strategies that can be more efficient in terms of resolution. This paper proposes an approach for ultrasound image enhancement that uses a hexagonal sampling scheme to display and process the images. The images were resampled on an interlaced grid. Interlaced sampling uses square pixels shifted half a pixel on alternate rows. Two types of hexagonal filters were designed and tested on ultrasound images: a statistical adaptive filter and an anisotropic diffusion filter. Results show improvements in signal-to-noise ratio and more natural representation of curved structures.

Variable transform sizes have been adopted by the emerging international standard of high-efficiency video coding (HEVC). The HEVC test model (HM) supports four different transform sizes in the range of 4×4 to 32×32 . The transform size is chosen based on the rate-distortion optimization techniques. Variable transform sizes have a coding performance superior to the traditional fixed block-size transform. Meanwhile, this causes high computational complexity at the encoders. We propose a fast transform size decision algorithm incorporating three methods: (1) transform bypass based on the SKIP mode detection, (2) content-based transform size decision, and (3) early termination of variable-sized transform. The proposed algorithm first checks coding information from nearby spatially and temporally coding units (CUs) or treeblocks and the parent CU in the upper depth level. Based on the coding information from nearby CUs, only a small number of transform sizes are selected in the procedure of transform size decision. Simulation results show that our algorithm can reduce the complexity of the transform procession by more than 55% with a negligible loss of coding efficiency.

Bilateral filtering is a nonlinear technique that reduces noise from images while preserving strong image edges. Due to the nonlinear nature of bilateral filtering, it is difficult to analyze the performance of the filter. We derive a closed-form equation of bilateral filtering for flat regions which shows the relationship between noise reduction and filtering parameters. This work explicitly shows that noise reduction depends on the ratio of the range parameter to the noise standard deviation, which confirms reported empirical observations. The derived result is a significant contribution for the analysis of bilateral filters toward estimating the optimal parameters for minimum mean square error. We demonstrate that the theoretical analysis presented is consistent with simulations.