This PDF file contains the editorial “Special Section Guest Editorial: Image/Video Quality and System Performance” for JEI Vol. 23 Issue 06

When a mobile phone camera is tested and benchmarked, the significance of image quality metrics is widely acknowledged. There are also existing methods to evaluate the camera speed. However, the speed or rapidity metrics of the mobile phone’s camera system has not been used with the quality metrics even if the camera speed has become a more and more important camera performance feature. There are several tasks in this work. First, the most important image quality and speed-related metrics of a mobile phone’s camera system are collected from the standards and papers and, also, novel speed metrics are identified. Second, combinations of the quality and speed metrics are validated using mobile phones on the market. The measurements are done toward application programming interface of different operating systems. Finally, the results are evaluated and conclusions are made. The paper defines a solution to combine different image quality and speed metrics to a single benchmarking score. A proposal of the combined benchmarking metric is evaluated using measurements of 25 mobile phone cameras on the market. The paper is a continuation of a previous benchmarking work expanded with visual noise measurement and updates of the latest mobile phone versions.

To understand the viewing strategies employed in a quality estimation task, we compared two visual tasks—quality estimation and difference estimation. The estimation was done for a pair of natural images having small global changes in quality. Two groups of observers estimated the same set of images, but with different instructions. One group estimated the difference in quality and the other the difference between image pairs. The results demonstrated the use of different visual strategies in the tasks. The quality estimation was found to include more visual planning during the first fixation than the difference estimation, but afterward needed only a few long fixations on the semantically important areas of the image. The difference estimation used many short fixations. Salient image areas were mainly attended to when these areas were also semantically important. The results support the hypothesis that these tasks’ general characteristics (evaluation time, number of fixations, area fixated on) show differences in processing, but also suggest that examining only single fixations when comparing tasks is too narrow a view. When planning a subjective experiment, one must remember that a small change in the instructions might lead to a noticeable change in viewing strategy.

Most current state-of-the-art blind image quality assessment (IQA) algorithms usually require process training or learning. Here, we have developed a completely blind IQA model that uses features derived from an image’s contourlet transform and singular-value decomposition. The model is used to build algorithms that can predict image quality without any training or any prior knowledge of the images or their distortions. The new method consists of three steps: first, the contourlet transform is used on the image to obtain detailed high-frequency structural information from the image; second, the singular values of the just-obtained “structural image” are computed; and finally, two new universal blind IQA indices are constructed utilizing the area and slope of the truncated singular-value curves of the “structural image.” Experimental results on three open databases show that the proposed algorithms deliver quality predictions that have high correlations against human subjective judgments and are highly competitive with the state-of-the-art.

The new video coding standard, high-efficiency video coding, adopts a quadtree structure to provide variable transform sizes in the transform coding process. The heuristic examination of transform unit (TU) modes substantially increases the computational complexity, compared to previous video coding standards. Thus, efficiently reducing the TU candidate modes is crucial. In the proposed similarity-check scheme, sub-TU blocks are categorized into a strongly similar case or a weakly similar case, and the early TU termination or early TU splitting procedure is performed. For the strongly similar case, a property called zero-block inheritance combined with a zero-block detection technique is applied to terminate the TU search process early. For the weakly similar case, the gradients of residuals representing the similarity of coefficients are used to skip the current TU mode or stop the TU splitting process. In particular, the computation time is further reduced because all the required information for the proposed mode decision criteria is derived before performing the transform coding. The experimental results revealed that the proposed algorithm can save ~64% of the TU encoding time on average in the interprediction, with a negligible rate-distortion loss.

Measuring the visual quality of printed media is important since printed products have an important role in everyday life. Finding ways to automatically predict the image quality has been an active research topic in digital image processing, but adapting those methods to measure the visual quality of printed media has not been studied often or in depth and is not straightforward. Here, we analyze the efficacy of no-reference image quality assessment (IQA) algorithms originally developed for digital IQA with regards to predicting the perceived quality of printed natural images. We perform a comprehensive statistical comparison of the methods. The best methods are shown to accurately predict subjective opinions of the quality of printed photographs using data from a psychometric study.

A recent development in the area of image and video quality consists of trying to incorporate aspects of visual attention in the design of visual quality metrics, mostly using the assumption that visual distortions appearing in less salient areas might be less visible and, therefore, less annoying. This research area is still in its infancy and results obtained by different groups are not yet conclusive. Among the works that have reported some improvements, most use subjective saliency maps, i.e., saliency maps generated from eye-tracking data obtained experimentally. Other works address the image quality problem, not focusing on how to incorporate visual attention into video signals. We investigate the benefits of incorporating bottom-up video saliency maps (obtained using Itti’s computational model) into video quality metrics. In particular, we compare the performance of four full-reference video quality metrics with their modified versions, which had saliency maps incorporated into the algorithm. Results show that the addition of video saliency maps improve the performance of most quality metrics tested, but the highest gains were obtained for the metrics that only took into consideration spatial degradations.

The goal of this work is to present a full-reference (FR) audio-visual quality metric. We performed three psychophysical experiments in order to obtain a better understanding of how audio and video components interact with each other and how these interactions affect the overall audio-visual quality. In experiment I, subjects evaluated the quality of videos (without any audio) compressed at different video bitrates. In experiment II, subjects evaluated the quality of audio (without any video) compressed at different audio bitrates. In experiment III, subjects evaluated the quality of videos (audio-visual signals), which had their audio and video components compressed at different bitrates. Based on the data gathered from these experiments, we obtain a set of subjective models for audio-visual quality. Inspired by these subjective models, we propose a set of FR audio-visual quality metrics composed of a combination of a video quality metric and an audio quality metric. The proposed metrics have good performance and present better results when compared to simple FR video quality metrics.

The vergence-accommodation conflict in holographic stereograms is investigated. The visual distortion and fatigue caused by the conflict are analyzed. A method for generating full-parallax holographic stereograms without vergence-accommodation conflicts is proposed. Two-dimensional spatial and spectral samplings are carried out on both the hologram and the reconstructed planes. The depth cues of three-dimensional object points are introduced in the iterative process of calculating subholograms with different spectral components. The stereogram is a combination of holographic elements (hogels), and each hogel is formed by performing a weighted summation of subholograms, where parallax images and depth information are used to select the constituent subholograms. A proof-of-principle experiment is carried out in an optical system based on a spatial light modulator. The results show that the improved full-parallax holographic stereogram can control the focusing depths of points and guarantee consistency between the vergence and accommodation distances. The influence of the size of the hogels on holographic imaging quality is also investigated.

Discrete cosine transform (DCT) is one of the major operations in image compression standards and it requires intensive and complex computations. Recent computer systems and handheld devices are equipped with high computing capability devices such as a general-purpose graphics processing unit (GPGPU) in addition to the traditional multicores CPU. We develop an optimized parallel implementation of the forward DCT algorithm for the JPEG image compression using the recently proposed Open Computing Language (OpenCL). This OpenCL parallel implementation combines a multicore CPU and a GPGPU in a single solution to perform DCT computations in an efficient manner by applying certain optimization techniques to enhance the kernel execution time and data movements. A separate optimal OpenCL kernel code was developed (CPU-based and GPU-based kernels) based on certain appropriate device-based optimization factors, such as thread-mapping, thread granularity, vector-based memory access, and the given workload. The performance of DCT is evaluated on a heterogeneous environment and our OpenCL parallel implementation results in speeding up the execution of the DCT by the factors of 3.68 and 5.58 for different image sizes and formats in terms of workload allocations and data transfer mechanisms. The obtained speedup indicates the scalability of the DCT performance.

Image-quality assessment measures are largely based on the assumption that an image is only distorted by one type of distortion at a time. These conventional measures perform poorly if an image includes more than one distortion. In consumer photography, captured images are subject to many sources of distortions and modifications. We searched for feature subsets that predict the quality of photographs captured by different consumer cameras. For this, we used the new CID2013 image database, which includes photographs captured by a large number of consumer cameras. Principal component analysis showed that the features classified consumer camera images in terms of sharpness and noise energy. The sharpness dimension included lightness, detail reproduction, and contrast. The support vector regression model with the found feature subset predicted human observations well compared to state-of-the-art measures.

Traditional Wyner–Ziv video coding (WZVC) structures require either intra (Key) or Wyner–Ziv (WZ) coding of frames. Unfortunately, keeping the video quality approximately constant implies drastic bit-rate fluctuations because consecutive frames of different types (Key or WZ) present significantly different compression performances. Moreover, certain scenarios severely limit rate fluctuation. This work proposes a WZVC scheme with low bit-rate fluctuations based on a symmetric coding structure. First, this work investigates the performance of a generic nonasymmetric distributed source coding structure, showing that the low-density parity-check accumulate channel decoding method is best suited. This is used as a basis to design a symmetric WZVC scheme in which every input video frame is divided into four parallel subframes through subsampling, and then the subframes are encoded by using a symmetric method. Compared with the traditional asymmetric WZVC scheme, the proposed scheme can achieve higher bit-rate stability over time, which is a great advantage to guarantee a reliable transmission in many wireless communication application environments in which bit-rate fluctuations are strongly constrained. Simulation results show the effectiveness of the proposed symmetric WZVC scheme in maintaining a steady bit rate and quality, as well as a quality comparison with the traditional WZVC scheme.

As an important problem in computer vision, saliency detection is essential for image segmentation, super-resolution, object recognition, and so on. We propose a saliency detection method for images. Instead of using contrast between salient regions and their surrounding areas, both cues from salient and nonsalient regions are considered in our study. Based on these cues, an improved multimanifold ranking algorithm is proposed. In our algorithm, features from multiple views are utilized and the different contributions of these multiview features are taken into account. Moreover, an iterative updating optimization scheme is explored to solve the objective function, during which the feature fusion is performed. After two-stage ranking by the improved multimanifold ranking algorithm, each image patch can be assigned a ranking score, which determines the final saliency. The proposed method is evaluated on four public datasets and is compared with the state-of-the-art methods. Experimental results indicate that the proposed method outperforms existing schemes both in qualitative and quantitative comparisons.

We evaluate the effects of transmission artifacts such as JPEG compression and additive white Gaussian noise on the performance of a state-of-the-art pedestrian detection algorithm, which is based on integral channel features. Integral channel features combine the diversity of information obtained from multiple image channels with the computational efficiency of the Viola and Jones detection framework. We utilize “quality aware” spatial image statistics to blindly categorize distorted video frames by distortion type and level without the use of an explicit reference. We combine quality statistics with a multiclassifier detection framework for optimal pedestrian detection performance across varying image quality. Our detection method provides statistically significant improvements over current approaches based on single classifiers, on two large pedestrian databases containing a wide variety of artificially added distortion. The improvement in detection performance is further demonstrated on real video data captured from multiple cameras containing varying levels of sensor noise and compression. The results of our research have the potential to be used in real-time in-vehicle networks to improve pedestrian detection performance across a wide range of image and video quality.

Due to the variance between subjects, there is usually ambiguity in intensity-based intersubject registration. The topological constraint in the brain cortical surface might be violated because of the highly convolved nature of the human cortical cortex. We propose an intersubject brain registration method by combining the intensity and the geodesic closest point-based similarity measurements. Each of the brain hemispheres can be topologically equal to a sphere and a one-to-one mapping of the points on the spherical surfaces of the two subjects can be achieved. The correspondences in the cortical surface are obtained by searching the geodesic closest points in the spherical surface. The corresponding features on the cortical surfaces between subjects are then used as anatomical landmarks for intersubject registration. By adding these anatomical constraints of the cortical surfaces, the intersubject registration results are more anatomically plausible and accurate. We validate our method by using real human datasets. Experimental results in visual inspection and alignment error show that the proposed method performs better than the typical joint intensity- and landmark-distance-based methods.

Iris images acquired under different conditions often suffer from blur, occlusion due to eyelids and eyelashes, specular reflection, and other artifacts. Existing iris recognition systems do not perform well on these types of images. To overcome these problems, we propose an iris recognition method based on robust principal component analysis. The proposed method decomposes all training images into a low-rank matrix and a sparse error matrix, where the low-rank matrix is used for feature extraction. The sparsity concentration index approach is then applied to validate the recognition result. Experimental results using CASIA V4 and IIT Delhi V1iris image databases showed that the proposed method achieved competitive performances in both recognition accuracy and computational efficiency.

We discuss new approaches to analyze laser-gated viewing data for nonline-of-sight vision with a frame-to-frame back-projection as well as feature selection algorithms. Although first back-projection approaches use time transients for each pixel, our method has the ability to calculate the projection of imaging data on the voxel space for each frame. Further, different data analysis algorithms and their sequential application were studied with the aim of identifying and selecting signals from different target positions. A slight modification of commonly used filters leads to a powerful selection of local maximum values. It is demonstrated that the choice of the filter has an impact on the selectivity i.e., multiple target detection as well as on the localization precision.

A saliency detection algorithm based on manifold-based similarity adaptation is proposed. The proposed algorithm is divided into three steps. First, we segment an input image into superpixels, which are represented as the nodes in a graph. Second, a new similarity measurement is used in the proposed algorithm. The weight matrix of the graph, which indicates the similarities between the nodes, uses a similarity-based method. It also captures the manifold structure of the image patches, in which the graph edges are determined in a data adaptive manner in terms of both similarity and manifold structure. Then, we use local reconstruction method as a diffusion method to obtain the saliency maps. The objective function in the proposed method is based on local reconstruction, with which estimated weights capture the manifold structure. Experiments on four bench-mark databases demonstrate the accuracy and robustness of the proposed method.

Camouflage is a challenging issue in moving object detection. Even the recent and advanced background subtraction technique, visual background extractor (ViBe), cannot effectively deal with it. To better handle camouflage according to the perception characteristics of the human visual system (HVS) in terms of minimum change of intensity under a certain background illumination, we propose an improved ViBe method using an adaptive distance threshold, named IViBe for short. Different from the original ViBe using a fixed distance threshold for background matching, our approach adaptively sets a distance threshold for each background sample based on its intensity. Through analyzing the performance of the HVS in discriminating intensity changes, we determine a reasonable ratio between the intensity of a background sample and its corresponding distance threshold. We also analyze the impacts of our adaptive threshold together with an update mechanism on detection results. Experimental results demonstrate that our method outperforms ViBe even when the foreground and background share similar intensities. Furthermore, in a scenario where foreground objects are motionless for several frames, our IViBe not only reduces the initial false negatives, but also suppresses the diffusion of misclassification caused by those false negatives serving as erroneous background seeds, and hence shows an improved performance compared to ViBe.

We present a framework for dynamic textures (DTs) recognition and localization by using a model developed in the text analysis literature: probabilistic latent semantic analysis (pLSA). The novelty is revealed in three aspects. First, chaotic feature vector is introduced and characterizes each pixel intensity series. Next, the pLSA model is employed to discover the topics by using the bag of words representation. Finally, the spatial layout of DTs can be found. Experimental results are conducted on the well-known DTs datasets. The results show that the proposed method can successfully build DTs models and achieve higher accuracies in DTs recognition and effectively localize DTs.

Compressed sensing (CS) is a new signal processing theory that provides an insight into signal processing. The CS theory has numerous potential applications in various fields, such as image processing, astronomical data analysis, analog-to-information, medical imaging, and remote sensing (RS) imagery. The CS theory is applied to RS video imagery. An RS video based on a compressed sensing (RS-VCS) framework with correlation estimation measurement is proposed, along with a block measurement correlation model and corresponding reconstruction. The linearized Bregman algorithm is used to solve the reconstruction model, and the performance of the RS-VCS framework is simulated numerically.

Conventional local-based color reconstruction for a white-RGB (WRGB) imaging system relies excessively on reference pixels. Therefore, it is sensitive to noisy interference. To address this issue, we propose an improved nonlocal fuzzy color segmentation-based color reconstruction hybrid approach for a WRGB imaging system. Unlike local-based approaches, we attempt to reproduce color information based on the statistical color distribution of the raw sensor data. According to the distribution analysis, the color distribution (histogram and cumulative distribution function) is close to that of the full-resolution image. However, brief histogram matching gives rise to zipper artifacts, which result from the multicombination of the red, green, and blue corresponding to one white. Therefore, a hybrid color segmentation is proposed to address this issue. The first step is a brief sorting-based color segmentation in the hue channel. Fuzzy-based color segmentation is then utilized to acquire more subregions in the proposed saturation space. Finally, fast histogram matching is carried out to obtain the full-color information for the white pixel for each region. Compared with state-of-the-art approaches, the proposed nonlocal hybrid approach is capable of significantly reducing the influence of noise with higher peak signal-to-noise ratios. Furthermore, according to the hybrid color segmentation, zipper artifacts are successfully avoided.

Signature authentication systems often have to focus their processing on acquired dynamic and/or static signatures descriptors to authenticate persons. This approach gives satisfactory results in ordinary cases but remains vulnerable against skilled forgeries. This is mainly because there is no relation between the signatory and his signature. We will show that the inclusion of the hand shape in the authentication process will considerably reduce the false acceptance rates of skilled forgeries and improve the authentication accuracy performances. A new online hand signature authentication approach based on both signature and hand shape descriptor is proposed. The signature acquisition is completely transparent, which allows a high level of security against fraudulent imitation attempts. Authentication performances are evaluated with extensive experiments. The obtained test results [equal  error  rate  (EER)=2%, genuine  acceptance  rate (GAR)=96%]confirm the efficiency of the proposed approach.

In previous studies on human face recognition, illumination pretreatment has been considered to be among the most crucial steps. We propose the illumination compensation algorithm two separated singular value decomposition (TSVD). TSVD consists of two parts, namely the division of high- and low-level images and singular value decomposition, which are implemented according to self-adapted illumination compensation to resolve the problems associated with strong variation of light and to improve face recognition performance. The mean color values of the three color channels R, G, and B are used as the thresholds, and two subimages of two types of light levels are then input with the division of the maximal mean and minimal mean, which are incorporated with light templates at various horizontal levels. The dynamic compensation coefficient is proportionately adjusted to reconstruct the subimages. Finally, two subimages are integrated to achieve illumination compensation. In addition, we combined TSVD and the projection color space (PCS) to design a new method for converting the color space called the two-level PCS. Experimental results demonstrated the efficiency of our proposed method. The proposed method not only makes the skin color of facial images appear softer but also substantially improves the accuracy of face recognition, even in facial images that were taken under conditions of lateral light or exhibit variations in posture.

Ruling gap ratios are an affine-invariant characterization of parallel ruling configurations in scanned documents. This report quantifies the advantage of simultaneous extraction of horizontal and vertical rulings. It demonstrates that every ruling gap ratio can be derived from a minimal set of basis ratios. The effect on the basis ratios of noise on the radial coordinates of individual rulings is analyzed and the dependence of basis-ratio variability on random-phase sampling noise is determined as a function of the spatial sampling rate. The analysis provides insight into already-presented small-scale experimental results on form classification and guidance for future work that requires the extraction of parallel lines from scanned or photographed images.

We present an interactive image deformation method which preserves the local shapes of salient objects in the concerned image during the deformation. The proposed method falls into the moving least squares (MLS) framework, but notably differs from the original MLS deformation method. First, a saliency-related distance is developed to replace the original Euclidean distance in the weight definition. Second, the original affine matrix is decomposed into a single rotation matrix and a symmetric matrix by using a singular value decomposition, then the free parameters of these matrices are interpolated according to the saliency information. Furthermore, for the line-based MLS deformation, the closed-form solution of weight cannot be found directly when using the proposed saliency-based distance. To address this problem, we propose a method using an exponential transformation to regulate the weight where the regulation factor is also correlated to saliency information. All these revisions lead a saliency-sensitive mapping which creates a deformation change in the nonvital parts of image while preserving the local shapes of salient parts. Experimental results show that the proposed deformation outperforms the original MLS deformation in terms of visual performance.

A recent trend in color halftoning is to explicitly control color overlapping, dot positioning, and dot coloring in different stages of the halftoning process. As feature-preserving multiscale error diffusion (FMED) shows its capability and flexibility in dot positioning in both binary and multilevel halftoning applications, it naturally becomes a potential candidate in the development of new color halftoning algorithms. This paper presents an FMED-based color halftoning algorithm developed based on the aforementioned strategy. In contrast to the algorithms that adopt the same strategy, the proposed algorithm has no bias for specific Neugebauer primaries in dot coloring and has no fixed scanning path to place dots in dot positioning. These features allow the algorithm to place dots of the correct colors on the right positions with less constraint. Hence, the algorithm is better able to preserve image features.

A spatial method of multistructure sampling based rotation-invariant uniform local binary pattern (named MsLBP^riu2) for classification of hyperspectral images is proposed. This method exploits the local property (micro-/macrostructure) of local image patches encoded in the classifier by considering a local neighboring structure around each central pixel and can well suppress the difference of rotational textures for each multicluster class. The proposed method is simple yet efficient for extracting isotropic and anisotropic spatial features from local image patches via different extended sampling on circular regions and elliptical ones with four different rotational angles. Furthermore, the rotation-invariant characteristic of extracted isotropic features is achieved by the inclusion of a rotation-invariant uniform LBP operator. Moreover, the proposed method becomes more robust with respect to the within-class variation. Finally, different classifiers, support vector machine, K-nearest neighbor, and linear discriminant analysis, are compared to evaluate MsLBP^riu2 and other feature sets/entropy-based query-by-bagging active learning. We demonstrate the performance of our approach on four different hyperspectral remote sensing images. Experimental results show that the new set of reduced spatial features has a better performance than a variety of state-of-the-art classification algorithms.

Blue noise sampling is a core component for a large number of computer graphic applications such as imaging, modeling, animation, and rendering. However, most existing methods are concentrated on preserving spatial domain properties like density and anisotropy, while ignoring feature preserving. In order to solve the problem, we present a new distance metric called mixture distance for blue noise sampling, which is a combination of geodesic and feature distances. Based on mixture distance, the blue noise property and features can be preserved by controlling the ratio of the geodesic distance to the feature distance. With the intention of meeting different requirements from various applications, an adaptive adjustment for parameters is also proposed to achieve a balance between the preservation of features and spatial properties. Finally, implementation on a graphic processing unit is introduced to improve the efficiency of computation. The efficacy of the method is demonstrated by the results of image stippling, surface sampling, and remeshing.

In three-dimensional video coding (3-DVC), it is reasonable to allocate different coding bits for multiview videos and associated depth maps because of their different characteristics to meet the bits’ restraints of bandwidth/storage. We first propose a virtual view average distortion model. Then, based on the quantization parameter (QP), the average distortion-QP models and sum bitrate-QP models are proposed to depict the average distortions and sum bitrates of the referenced views for multiview videos and depth maps. Finally, the 3-DVC bit allocation problem is converted as a constrained optimization problem, which is solved by a genetic algorithm to search for the optimal QP pair. Experimental results demonstrate the effectiveness of our proposed models. Since the bit allocation scheme takes the performance of synthesized view and bitrate utilization into consideration, the absolute difference between the constraint and the actual coding bitrates (referred to as “rate inaccuracy”) of the proposed method is only 7.405% on average, which greatly outperforms the fixed 5 ∶ 1 ratio-based method with a rate inaccuracy of 19.103% and the planar model-based method with a rate inaccuracy of 20.556%. Compared with these two methods, our proposed method can achieve a maximum 1.951-dB gain under the same bitrates constraint.

A simple power-logarithm histogram modification operator is proposed to enhance digital image contrast. First, a logarithm operator reduces the effect of spikes and transforms the image histogram into a smoothed one that approximates a uniform histogram while retaining the relative size ordering of the original bins. Then, a power operator transforms this smoothed histogram into one that approximates the original input histogram. Contrast enhancement is then achieved by using the cumulative distribution function of the resulting histogram in a standard lookup table-based histogram equalization procedure. The method is computationally efficient, independent of image content, and (unlike most existing contrast enhancement algorithms) does not suffer from the occurrence of artifacts, overenhancement and unnatural effects in the processed images. Performance comparisons with several state-of-the-art contrast enhancement algorithms show that the proposed method achieves appreciable contrast enhancement under diverse conditions.

Video-frame registration is a critical step in the video super-resolution process. However, many existing registration methods can handle only small local neighborhood movements or overall affine transformations. This limits the reconstruction of fast moving or significantly deforming objects. We propose a super-resolution registration method for video reconstruction. Considerable effort has been directed toward single-video and multivideo super-resolution methods. We aim to obtain a higher registration accuracy that maximally employs video frames to reconstruct the current frame, particularly for moving or deforming objects. To this end, we provide a content-based registration algorithm based on a propagation matching algorithm and the Lucas–Kanade method. The super-resolution step is implemented using robust iterative minimization. We compare our algorithm to others and demonstrate that our algorithm achieves high registration accuracy and more effectively reconstructs fast moving and significantly deforming objects.

A three-dimensional (3-D) spatiotemporal prediction-error filter (PEF) is used to enhance foreground/background contrast in (real and simulated) sensor image sequences. Relative velocity is utilized to extract point targets that would otherwise be indistinguishable with spatial frequency alone. An optical-flow field is generated using local estimates of the 3-D autocorrelation function via the application of the fast Fourier transform (FFT) and inverse FFT. Velocity estimates are then used to tune in a background-whitening PEF that is matched to the motion and texture of the local background. Finite impulse response (FIR) filters are designed and implemented in the frequency domain. An analytical expression for the frequency response of velocity-tuned FIR filters, of odd or even dimension with an arbitrary delay in each dimension, is derived.

Image steganography delivers secret data by slight modifications of the cover. To detect these data, steganalysis tries to create some features to embody the discrepancy between the cover and steganographic images. Therefore, the urgent problem is how to design an effective classification architecture for given feature vectors extracted from the images. We propose an approach to automatically select effective features based on the well-known JPEG steganographic methods. This approach, referred to as extreme learning machine revisited feature selection (ELM-RFS), can tune input weights in terms of the importance of input features. This idea is derived from cross-validation learning and one-dimensional (1-D) search. While updating input weights, we seek the energy decreasing direction using the leave-one-out (LOO) selection. Furthermore, we optimize the 1-D energy function instead of directly discarding the least significant feature. Since recent Liu features can gain considerable low detection errors compared to a previous JPEG steganalysis, the experimental results demonstrate that the new approach results in less classification error than other classifiers such as SVM, Kodovsky ensemble classifier, direct ELM-LOO learning, kernel ELM, and conventional ELM in Liu features. Furthermore, ELM-RFS achieves a similar performance with a deep Boltzmann machine using less training time.

Due to various reasons (e.g., poor quality of image, sudden changes in light conditions, occlusions, or objects with similar colors), holes may appear in the moving object region when performing moving object segmentation. How to remove these holes is a crucial problem in high-quality segmentation. A moving object segmentation approach combining symmetrical differencing and joint bilateral filtering is proposed. One important aspect of our approach is that joint bilateral filtering is used to eliminate/reduce the holes in the moving object region, which makes the extracted moving object more accurate and complete, in many cases in contrast to the reference temporal difference approaches. The approach first performs block-based symmetrical differencing to obtain coarse moving object regions. Then, a joint bilateral filter that uses the current gray image as a guide image is adopted to smooth the difference image. Next, edge detection is performed on the smoothed difference image and the current gray image, respectively, to get the edges of moving objects. Finally, the undesired background is distinguished from the moving object and is cut off by a postprocessing module. In addition, the fast joint bilateral filters that can be used as substitutes for the classical joint bilateral filter are discussed. Experimental results show that this approach can effectively fill the holes in the moving object region and provide an improvement in accuracy of moving object segmentation in many cases.

Recently, multitask learning was applied to visual tracking by learning sparse particle representations in a joint task, which led to the so-called multitask tracking algorithm (MTT). Although MTT shows impressive tracking performances by mining the interdependencies between particles, the individual feature of each particle is underestimated. The utilized L_1,q norm regularization assumes all features are shared between all particles and results in nearly identical representation coefficients in nonsparse rows. We propose a composite sparse multitask tracking algorithm (CSMTT). We develop a composite sparse model to formulate the object appearance as a combination of the shared feature component, the individual feature component, and the outlier component. The composite sparsity is achieved via the L_1,∞ and L_1,1 norm minimization, and is optimized by the alternating direction method of multipliers, which provides a favorable reconstruction performance and an impressive computational efficiency. Moreover, a dynamical dictionary updating scheme is proposed to capture appearance changes. CSMTT is tested on real-world video sequences under various challenges, and experimental results show that the composite sparse model achieves noticeable lower reconstruction errors and higher computational speeds than traditional sparse models, and CSMTT has consistently better tracking performances against seven state-of-the-art trackers.

This paper presents an optimization method to reduce blocking artifacts in JPEG images by utilizing the image gradient information. A closed-form solution is derived for the optimization method. To address the computational feasibility aspect of the large matrices involved in the closed-form solution, a sliding window approach is devised. The performance of the developed method is compared with several blocking artifacts reduction methods in the literature and also with the deblocking filter deployed in high efficiency video coding by examining the three measures of peak signal-to-noise ratio, generalized block-edge impairment metric (M_GBIM), and structural similarity. The comparison results indicate the effectiveness of the introduced method in particular for low bit-rate JPEG images.