Crowd density estimation is an important part of intelligent crowd monitoring. However, there are still many problems in density estimation due to the complexity of crowd scenes. Aiming at the high-density scenes with varied scales, we present a method based on cross-adversarial loss and global feature for crowd counting, so as to achieve the purpose of capturing more feature details and reducing the impact of background noise more effectively. First, we use the cross-adversarial loss to generate the residual map, which makes use of the consistency between different scales and solves the homogenization problem of fused density map. Then, we extract large-range context information and focus on key information in global spatial features for the generation of a residual map. Finally, the high-resolution density map is used to estimate the crowd counting. Experiments on three datasets confirm that the proposed method has good adaptability in scenes with obvious distribution change, not just in extracting high-quality features for density map estimation but also for accurate crowd counting.

Object detection for blind zones is critical to ensuring the driving safety of heavy trucks. We propose a scheme to realize object detection in the blind zones of heavy trucks based on the improved you-only-look-once (YOLO) v3 network. First, according to the actual detection requirements, the targets are determined to establish a new data set of persons, cars, and fallen pedestrians, with a focus on small and medium objects. Subsequently, the network structure is optimized, and the features are enhanced by combining the shallow and deep convolution information of the Darknet platform. In this way, the feature propagation can be effectively enhanced, feature reuse can be promoted, and the network performance for small object detection can be improved. Furthermore, new anchors are obtained by clustering the data set using the K-means technique to improve the accuracy of the detection frame positioning. In the test stage, detection is performed using the trained model. The test results demonstrate that the proposed improved YOLO v3 network is superior to the original YOLO v3 model in terms of the blind zone detection and can satisfy the accuracy and real-time requirements with an accuracy of 94% and runtime of 13.792 ms   /  frame. Moreover, the mean average precision value for the improved model is 87.82%, which is 2.79% higher than that of the original YOLO v3 network.

Human action recognition based on skeleton currently has attracted a wide range of attention. The structure of skeleton data exists in the form of graph, thus most researchers use graph convolutional networks (GCN) to model skeleton sequences. However, the graph convolution network shares the same weight for all neighbor nodes and relies on the connection of graph edges. We introduce a method, a spatial–temporal graph attention networks (ST-GAT), to overcome the disadvantages of GCN. First, the ST-GAT defines the spatial–temporal neighbor nodes of the root node and the aggregation function through the attention mechanism. The adjacency matrix is only used in GAT to define related nodes, and the calculation of association weight is dependent on the feature expression of nodes. Then ST-GAT network attaches the obtained attention coefficient to each neighbor node to automatically learn the representation of spatiotemporal skeletal features and output the classification results. Extensive experiments on two challenging datasets consistently demonstrate the superiority of our method.

In recent years, helping individuals with color vision deficiency to distinguish confusing colors in digital images, which is called Daltonization, is a hot topic. However, a number of Daltonization methods have a color reduction problem that causes unnatural image colors for normal color vision observers and those with anomalous trichromatic color vision deficiencies. A color-enhancing algorithm is proposed to make up for the shortage in color naturalness. Based on the conclusion in previous studies that colors confused in the same type of color blindness are approximately straight lines (called confusion lines) on the u^′v^′ chromaticity plane and all of the confusion lines intersect at the same point (called the confusion point), we propose a polar coordinate transformation based on intersection points of confusion lines, which can transform chromaticity information into perceptually sensitive and perceptually insensitive color blindness information. From the two color-blind sensitive information of the Daltonized image and the one color-blind insensitive information of the original image, the three-dimensional information can be combined to obtain an enhanced image. The enhanced image has a similar color appearance of the Daltonized image under the perspective of dichromats and has a more natural and colorful color appearance under the perspective of anomalous and normal trichromats. In addition, we propose a lightness modification to reduce lightness errors between the enhanced images and the Daltonized images. The quantitative evaluation shows that the method proposed is effective but sacrificing a small amount of color contrast of the Daltonized images.

Predicting bounding box with higher intersection over union (IoU) is one of the most important issues in many computer vision tasks. The ℓ_n-norm loss and IoU-based loss are two conventional approaches to guide a training process in recent methods. However, the optimization direction of ℓ_n-norm loss is not exactly the same as maximizing the metric. In addition, IoU-based loss suffers from some inevitable disadvantages due to the direct addition of IoU. According to the shape, size, and position properties, we design a mixed geometric (MG) regression loss to increase the similarity and the overlapping area of two bounding boxes. The shape is described by the cosine similarity of diagonal vectors, the size is measured by the length or width of the boxes, and the location is calculated by the center positions of the boxes. Simulation experiments verify that the proposed MG loss can achieve competitive convergence speed and regression accuracy. By introducing the state-of-the-art models in object detection, experiments are carried out on a well-known benchmark dataset, and the results demonstrate the effectiveness of our method in object detection.

Color constancy algorithms can provide us with illuminant invariant descriptions for a scene, and it is often accomplished by illuminant estimation. Most statistics-based methods estimate the illuminant color from the information provided by all pixels of an image. However, this research reveals that, for most images, the color of many pixels is quite different from the illuminant, and these pixels may severely trim the performance of statistics-based methods. Based on this fact, we propose a color constancy algorithm that finds a subset of image pixels with r, g components similar to those of the illuminant through a shallow neural network, and this subset of pixels is called illuminant close pixels (ICPs). Then the illuminant color is estimated from these pixels by some statistics-based methods. The proposed method has been evaluated and investigated on two benchmark datasets. Compared to using all pixels in an image, these statistics-based methods have been efficiently improved using ICPs.

Multiview learning is an important method and widely used for feature fusion in the fields of image process or big data analysis. Determining how to integrate compatible and complementary information from multiple views is a crucial and challenging task. We present a multiview feature fusion optimization method for image retrieval based on matrix correlation. This method first extracts four view features (Gist, histogram of color, pyramid histogram of oriented gradients, and multitrend structure descriptor) from the image. Then these features are, respectively, converted to different graph Laplacian matrices through local embedding. Third, a multiview feature alternating optimization process is constructed using matrix correlation statistics that adaptively combines the different view feature maps to a unified, low-dimensional embedding. Finally, the fusion feature is used for image retrieval experiments. Various experimental results show that the proposed algorithm is an effective method.

Biometric identification from three-dimensional (3-D) face surface characteristics has become popular. Traditional face recognition methods have achieved very high recognition accuracy under controlled environments. However, 3-D face recognition technology still faces a great challenge for facial expressions and missing data caused by pose variations or occlusions. The mesh-based scale-invariant feature transform (SIFT)-like method is one of the most effective methods to deal with these issues. However, eliminating edge response has not been considered in this kind of method, which affects recognition performance and computation cost. To address this problem, a mesh edge point filter is proposed to remove edge keypoints. For feature description, a local descriptor that is composed of the histogram of geometric shapes and the histogram of shape index is designed to describe local shapes of keypoints. A complete mesh-based SIFT-like framework for 3-D face recognition is also presented. Results from experiments based on the Bosphorus and face recognition grand challenge v2.0 databases show that the proposed method is effective for expression variations, occlusions, and pose variations. Compared with several state-of-the-art methods, the proposed method has obtained a great compromise between computational complexity and recognition performance.

Development of computer vision technologies has been widely used to increase the level of agricultural intelligence. Crop counting, an application of image counting, plays a fundamental role in agricultural information automation. However, the complex cotton field environment is likely lead to incorrect detection of the target position or fragmentation of the segmentation results, resulting in a decrease in counting accuracy. Despite this, computer vision technologies have shown great potential to effectively solve this task. To solve the problem of multimode cotton boll counting in a complicated environment, an in-field cotton boll counting algorithm based on density classification is proposed. First, the algorithm encodes the global context information with a density level classification estimator. Then, the input images are converted into high-dimensional feature maps by density map estimator with a multicolumn structure. Finally, through the feature fusion neural network, the classification information is combined with high-dimensional feature maps to generate a high-quality density map, and then the cotton bolls are counted. In particular, we collected and labeled a cotton boll counting dataset with 758 high-resolution images for experiment and comparison, which can be divided by different environmental conditions and observations sites. In addition, the relationship of cotton yield against the cotton bolls counting can be assessed by adopting our proposed algorithm. Experimental results demonstrate that the proposed algorithm achieves a lower counting error and better effectiveness and robustness than other comparative algorithms.

Tracking objects with the scale variations quickly is a challenging problem in visual tracking. Most existing methods estimate the scale of the object with an exhaustive search strategy, which needs large calculations but with less improvement. We propose to use twofold weighted phase correlation with the kernel (WPCK) to simultaneously estimate the translation and scale for the object during tracking. Besides, from the linear phase correlation, a weighted kernel phase correlator is first introduced to improve tracking performance. It extends the filter into a nonlinear space, which is more robust to signal noises and distortions. To efficiently locate the object in a large three-dimensional space for real-time running, we formulate tracking problem into two independent subproblems: the translation offset and scale offset. The mechanism is: through the log-polar transform of the amplitude spectrum, the scale estimation can be performed with phase correlation, which is also utilized to solve the translation offset. Thus, the accurate object status can be achieved effectively by twofold WPCK. Extensive experimental results on OTB2013, OTB100, and UAV123 datasets reveal that the proposed method has superior performance gains over several state-of-the-art trackers in terms of accuracy, robustness, and speed.

A motion energy image (MEI) is a spatial template that collapses regions of motion into a single image in which more moving pixels are brighter than others. The forward single-step history image (fSHI) is a spatiotemporal template that shows the presence and direction of motion. Each video can be described using these templates. Recently, the popularity of deep learning architectures for human activity recognition encourages us to explore the effectiveness of combining them with these templates. Hence, three new methods are introduced to convert the problem of human activity recognition in video into image templates classification. In method 1, each video is split into N groups of consecutive frames, and the MEI is computed for each group. Transfer learning with the fine-tuning technique is used for classifying these templates. fSHIs or spatiotemporal templates are used for classification in method 2, similar to method 1. Fusing the two streams of these templates is introduced as method 3. Among these methods, method 3 outperforms the others and is called the proposed method. It achieves recognition accuracies of 92.60% and 93.40% for the UCF Sport and UCF-11 action datasets, respectively. Also, it is compared with state-of-the-art approaches, and the results show that the proposed method has the best performance.

We describe the improvements of the content-based image retrieval (CBIR) system using a fuzzy class membership for the natural-color images. The fuzzy class membership-based retrieval (CMR) framework has shown promising improvements on texture databases by exploiting confidence in classification using a multilayer perceptron (MLP). CMR is known to improve the average precision of retrieval along with modest variance, and the framework is not restricted to any particular feature set. However, their efficacy is not known for natural colored images. In the proposed approach, we have added a new classifier, radial basis function network, in place of MLP in the CMR framework. We show a way to adapt a new classifier in the fuzzy CMR framework. Comparison with state-of-the-art CBIR systems shows that the proposed modifications have an edge over its competition in terms of precision for four popular image databases: viz. Corel-1k, Corel-5k, Corel-10k, and CIFAR-10.

Current self-supervised monocular methods only learn effectively by imposing consistency constraints without relying on any geometric constraints or ground truth depth constraints, which makes the accuracy of the estimation result suboptimal. Compared with the monocular algorithm, the stereo matching network usually follows the geometric process of the traditional stereo algorithm, which makes the estimation result more accurate. Inspired by these findings, we proposed a weakly supervised monocular learning approach that makes use of the disparity maps generated by the self-supervised stereo matching model as the “ground truth” labels to train a self-supervised monocular depth estimation model. To obtain more accurate ground truth labels, we improve the layer of geometry and context in self-supervised deep stereo regression by replacing the 3D convolutional layer with a guided aggregation layer. The design can also reduce computational costs and memory consumption. Then, we build our weakly supervised monocular model by improving the U-Net model and designing a loss function composed of a weakly supervised cost and a self-supervised cost. The estimation results obtained using our model outperform those of the existing self-supervised depth estimation methods under the same training conditions on the challenging KITTI dataset, and the results can easily be generalized to the Cityscapes dataset.

It is a challenge to remove the cluttered background in research of hand gesture images. The popular method, image semantic segmentation, is still not efficient enough to deal well with fine-grained image background removal due to insufficient training samples. We are the first to propose a background removal method based on a conditional generative adversarial network (CGAN). With CGAN, our method is designed to translate the images with backgrounds to the ones without backgrounds. The proposed method does not rely on the traditional image-to-semantics complex processing, and instead, performs an image-to-image task. The image is generated without background, and a discriminator decides whether backgrounds exist in output images. With an iterative training generator and discriminator, it is easy to fulfill two goals: (i) improving the discriminator’s ability to recognize whether the generated images have backgrounds and (ii) enhancing the generator’s ability to remove backgrounds. For our study, a large number of gesture images were collected and simulated to conduct experiments. The results demonstrate that the proposed method achieves remarkable performance in background removal for different gesture images. The training is robust, and the simple network has generalization ability among different hand gestures.

Blind deconvolution is known as a challenging low-level vision problem due to the diverse blur scenarios in real-world imaging. Another attempt is made with critical thoughts on existing image priors for nonparametric blur kernel estimation, proposing an alternative approach to blind deconvolution via complementarily structure-aware image smoothing (CSIS). Similar to most state-of-the-art blind deblurring methods, the proposed approach partly builds on the naïve L0-based sparse model, but the core contribution here is to additionally advocate a type of redescending potential (RDP) functions as a more elegant way for boosting blind deblurring. With the RDP element, the new approach is capable of easily achieving the discrimination between clear and blurred images. Meanwhile, the performance of the proposed method can be better ensured due to the complementary smoothing behavior induced by the RDP functions. Specifically speaking, it is known that L0-based smoothing plays a critical role in pursuing salient step-edges from blurry images especially in the early period of blur kernel estimation, whereas the RDP-based smoothing is found particularly significant in the later stage when the fine pursuit of salient roof-edges or ramp-edges are dominantly critical for more precise and robust kernel estimation. In this sense, the proposed CSIS-based blind deblurring algorithm is more intuitive than previous L0-based methods. Not only that, numerous experimental results on benchmark blurred images, no matter synthetic or realistic, also demonstrate the comparable or even better performance of the proposed approach in terms of both effectiveness and robustness.

The fusion of depth acquired actively with the depth estimated passively proved its significance as an improvement strategy for gaining depth. This combination allows us to benefit from two sources of modalities such that they complement each other. To fuse two sensor data into a more accurate depth map, we must consider the limitations of active sensing such as low lateral resolution while combining it with a passive depth map. We present an approach for the fusion of active time-of-flight depth and passive stereo depth in an accurate way. We propose a multimodal sensor fusion strategy that is based on a weighted energy optimization problem. The weights are generated as a result of combining the edge information from a texture map and active and passive depth maps. The objective evaluation of our fusion algorithm shows an improved accuracy of the generated depth map in comparison with the depth map of every single modality and with the results of other fusion methods. Additionally, a visual comparison of our result shows a better recovery on the edges considering the wrong depth values estimated in passive stereo. Moreover, the left and right consistency check on the result illustrates the ability of our approach to consistently fuse sensors.

In the tracking-by-detection scheme of multiple object tracking (MOT), the data association process in which existing tracking data and new detections are matched over time is very important. A framework is proposed to solve the data association problem in MOT in scenarios where there are potential target interactions and occlusions in crowded environments. This framework consists of an input layer and an association layer. The input layer is an end-to-end feature-map extraction model that incorporates a simplified Siamese convolutional neural network, which effectively distinguishes similar objects based on their appearance and motion. The association layer is composed of a bidirectional gated recurrent unit network with three layers of fully connected networks (FCNs), whose outputs are fed into the FCNs and transformed into an association matrix that reflects the matching scores between the detections and existing tracks. The matrix is then used to minimize the loss of the framework. The experimental results show that the proposed framework demonstrates outstanding performance for MOT, with its accuracy and precision in MOT reaching values as high as 26.1% and 71.2%, respectively.

This erratum corrects errors in tables and figures in the published paper.