Illegal construction should be detected as early as possible as it can damage the environment and economy. However, the existing methods for detecting illegal construction can be improved in terms of their detection cycles, accuracy, and speed. Moreover, there are relatively few valuable real-world image datasets for detecting illegal construction. To address these issues, a high-precision real-time detection model named YEMNet and a new large-scale dataset for detection of illegal construction objects (ICOS) are proposed herein. Our YEMNet is based on the You Only Look Once v3 object detection model; this model adopts a lightweight convolutional neural network called “EfficientNet” as the backbone for feature extraction. Then, YEMNet employs a new activation function Mish outside the backbone to achieve efficient optimization and strong generalization, thereby improving the recognition accuracy for ICOS in complicated scenes. Our proposed dataset comprises 15 categories and 13,701 photographs of ICOS captured under different conditions concerning weather, lighting, and natural scenes. Extensive experiments on the proposed dataset show that YEMNet achieves a mean average precision of 91.41% with fewer parameters, thereby outperforming state-of-the-art object detectors. Our dataset and code are available at https://github.com/king-king-king/ICOS-Dataset.

Recently, the demand for the generation, sharing, and storage of massive amounts of multimedia information—especially in the form of images—from different intelligent devices and sensors has increased drastically. This introduces issues including the illegal access and fraudulent usage of this information as well as other security concerns. Watermarking consists of embedding a watermark design in a digital cover and then later extracting it to provide a solution for ownership conflict and copyright violation issues involving the media data. Presently, in watermarking, the use of deep-learning approaches is incredibly beneficial due to their accuracy, superior results and strong learning ability. We present a comprehensive review of watermarking techniques in deep-learning environments. We start with basic concepts of traditional and learning-based digital watermarking; we later review the popular deep-learning model-based digital watermarking methods; then, we summarize and compare the most recent contribution in the literature; finally, we highlight obfuscation challenges and further research directions.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

LM and TD either did not respond directly or could not be reached.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

GW, YL, and NW either did not respond directly or could not be reached.

The Editor-in-Chief and the publisher have retracted this article, which was submitted as part of a guest-edited special section. An investigation uncovered evidence of systematic manipulation of the publication process, including compromised peer review. The Editor and publisher no longer have confidence in the results and conclusions of the article.

JS and BS either did not respond directly or could not be reached.

The K-means clustering algorithm is affected by the initial cluster center, resulting in a low accuracy of the clustering results. The standard flower pollination algorithm (FPA) has slow convergence and low optimization accuracy in the later stage. Therefore, the FPA is improved, and k-means is optimized accordingly. First, a random reverse learning strategy is used to uniformly distribute the population; second, the dynamic transition probability is used to balance the search mode to improve the overall performance of the algorithm; third, the nonlinear inertia weight parameter is introduced into the global search process to improve the global exploration ability; fourth, the optimal individual improves the diversity of the population while decreasing the probability of the algorithm failing. Six standard test functions are used to test the performance of improved flower pollination algorithm (IFPA), and the results show that IFPA is better than FPA in convergence speed and search optimization accuracy. The experimental comparative analysis of k-means cluster optimization based on improved flower pollination algorithm (IFPA-KM) on the University of California Irvine dataset shows that compared with k-means and FPA-KM, IFPA-KM improves the accuracy of clustering and has better stability.

The challenges in today’s medicine are progressively more related to the application of artificial intelligence and supervised learning techniques. Optical coherence tomography (OCT) is a noninvasive imaging technology used to obtain high-resolution cross-sectional images of the retina. The layers within the retina can be differentiated and retinal thickness can be measured to facilitate early detection and diagnosis of retinal diseases and conditions. This research paper is aimed at exploring different possibilities of applying deep learning, specifically convolutional neural networks in retinal diseases. During the research, several different architectures—AlexNet, VGG, Inception, and residual network, were evaluated, and the convolutional neural network that proved to be the most successful in the classification was based on the Inception architecture. Hyperparameter tuning was applied as the main method to find the most optimal solution. The key contributions of this research refer to the analysis of different architectures that can be applied in the classification of retinal diseases based on OCT images, as well as the evaluation of the test set obtained by comparing different models with different hyperparameters. This research yielded the best results obtained with Inception1, when training by means of the root mean square propagation optimizer with a batch size of 32, learning rate of 1e ^{− 6}, momentum of 0.99, and L2 regularization rate of 0.001. This model achieved an accuracy of 0.95528. In the conclusion of the paper, the advantages of the proposed and implemented solutions were discussed, and a proposal for further improvements was proposed.

Traditional zero-shot object-detection algorithms detect images of untrained classes in the model with the help of semantic embedding. However, these approaches may perform poorly due to the limitations of fixed semantic embedding. Given that fixed semantic attributes lead to a lack of generalization capabilities in the model, a semantic enhancement mechanism is proposed to update the semantic embedding, which is used to serve the needs of the visual space. Specifically, considering that the original semantic space is not enough to construct a visual-semantic mapping relationship, an augmented semantic embedding (ASE) approach is designed to supplement semantic attribute information. Then, a semantic channel attention mechanism is used to adjust the ASE. The adjustment strategy retains adequate attribute information, which is highly relevant to visual features. Finally, to alleviate the domain shift problem, a clustering association strategy is introduced to establish an inferred relationship, which ensures that the predictor is generalized to the unseen domain during training. The superiority of the proposed method is demonstrated by the MS-COCO and PASCAL VOC datasets.

With the continuous development of intelligent unmanned aerial vehicles reconnaissance, single-object detection or semantic segmentation can no longer meet the diversified requirements, while simple model stacking will cause the model to be too complex, which will seriously affect the real-time running effect. We propose a visual scene understanding algorithm based on a multitask learning network with an encoder–decoder structure. First, the efficient classification network VoVNet is selected as the feature-sharing network to obtain multiscale coding features. Second, based on the one-stage anchor-free object detector, the feature screening supplementary module implements feature interaction with segmented characters to enhance the detection capability of potential targets. Then for semantic segmentation and depth estimation pixel-level classification tasks, a cascaded chained residual pooling module is used as a parameter-sharing decoder, through the parameter-sharing mechanism to reduce the repeated decoding process to ensure the running speed. Finally, to improve the generalization ability of the model, a general-purpose dataset was constructed and network training was carried out based on the idea of knowledge distillation. Experiments on the dataset show that the performance of the multitask learning network can reach the mainstream algorithm level in semantic segmentation and depth estimation branches, whereas the object detection branch is better in multitarget recall rate, and the average time of each subtask is 39 ms, which meets the requirements of real-time performance and proves the effectiveness of the multitask network.

Human action recognition is a research hotspot in the field of computer vision. Focusing on the problem of similar action recognition, we propose an improved two-stream adaptive graph convolutional network for skeleton-based action recognition, which incorporating a multiscale temporal convolutional network and a spatiotemporal excitation network. Using the multiscale temporal convolutional network, the temporal information can be effectively extracted by dilated convolution at different scales so as to broaden the width of the temporal network and extract more temporal features with slight difference between categories at the same time. By utilizing the spatiotemporal excitation network, the input features can be obtained through channel pooling to form single-channel features for two-dimensional convolution, by which important spatiotemporal information can be excited and the role of local nodes in similar actions can be effectively enhanced. Extensive tests and ablation studies on the three large-scale datasets, NTU-RGB+D60, NTU-RGB+D120, and Kinetics-Skeleton, were conducted. Our model outperforms the baseline by 7.2% and the state-of-the-art model by 4% in the similar action recognition on NTU-RGB+D60 dataset on average, which demonstrates the superiority of our model.

To balance noise reduction and texture details of low-dose computed tomography (LDCT) images during deep learning training, we propose a network combining edge features and a pyramid structure for LDCT denoising (PLEDNet). PLEDNet contains three states: superficial feature acquisition, deep feature acquisition, and feature confluence. First, the superficial feature acquisition stage uses a simple convolutional block to perform feature extraction on the input LDCT image. Second, in the deep feature extraction phase, one branch uses pyramid pooling to extract multiscale features, and the other uses the trainable Laplacian of the Gaussian module to extract edge detail features. Finally, the feature fusion phase uses a fusion attention module to adaptively fuse multiscale and edge features. The results of the experiments demonstrate that the method not only enhances various relative objective indicators but also effectively represses the noise in LDCT images while fully conserving the image texture details, which has great advantages in terms of visual effects.

The performance of abnormal event detection has been greatly improved in recent years. However, due of the problems of low resolution and few labeled training samples, it is difficult to apply in practice. To address these issues, we propose a few-shot scene-adaptive abnormal event detection method. First, to restore high-quality video from low-quality video, we propose a weighted recurrent backprojection network (WRBPN). WRBPN selectively extracts supplementary information from adjacent frames based on their similarity, which is beneficial to the reconstruction of the target frame and suppresses the interference of irrelevant frames. Second, we propose an abnormal event detection network to detect abnormal events in high-quality surveillance videos. The network extracts multiscale features to obtain more discriminative representations and uses variance attention to suppress the interference of background noise. On this basis, the video frame is predicted, and abnormal event detection is performed according to the prediction error. Finally, in order to quickly adapt the model to new scenarios, we propose a metalearning-based approach. During the training process, the distribution of model parameters is learned, and the specific parameters of the model are obtained via distribution sampling. Experiments on benchmark datasets demonstrate the effectiveness of our method.

Adversarial examples are used to evaluate the robustness of convolutional neural networks (CNNs) to input perturbations. Researchers have proposed different types of adversarial examples that attack CNNs to fool them. These attacks pose a serious threat to applications that use deep neural networks. Existing methods for adversarial image generation struggle in maintaining a balance between attack success rate and imperceptibility (measured in terms of ℓ₂-norm) of the generated adversarial examples. Recent sparse methods for this problem focus on limiting the number of pixels in an image but do not cater to the overall imperceptibility of the adversarial images. To address these problems, we introduce adversarial attacks based on K-singular value decomposition sparse dictionary learning. The dictionary is learned using feature maps of the targeted images from the first layer of CNN. The proposed method is evaluated in terms of attack success rate and ℓ₂-norm. The extensive experimentation shows our attack achieves a high success rate while maintaining a low imperceptibility score compared to state-of-the-art methods.

The occlusion problem is a major challenge in the field of computer vision. Synthetic aperture imaging (SAI) is often used for surface reconstruction of occluded objects. However, SAI usually relies on high-speed information transmission devices. In addition, a large amount of information in the scene is lost, when handling low-resolution input images. This limitation results in unclear reconstructed regions in the synthetic aperture image and thus hinders the application of SAI in downstream tasks. We propose a multi-view super-resolution SAI method. It aims to generate high-resolution synthetic aperture images using images acquired by few of low-resolution acquisition devices. The main contributions of this paper are: (1) a multi-view super-resolution algorithm is proposed. It can generate clear synthetic aperture images in an array with a limited number of cameras. (2) By exploiting the correlation between views, the proposed algorithm can generate super-resolution synthetic aperture images with more accurate image structure and sharper image edges. (3) A feature extraction module is proposed. It can effectively extract the complementary relationship between pictures from different perspectives. The experimental results show that the proposed method can generate a reconstructed image of the occluded object surface with clear edges and accurate structure. Compared to conventional SAI, our method improves 5.7 % / 21.1 % on peak signal-to-noise ratio (PSNR)/structure similarity index measure (SSIM) and 4.4 % / 9.2 % on PSNR/SSIM respectively on two datasets compared to other state-of-the-art super-resolution methods.

The worldwide spread of the coronavirus disease 2019 (COVID-19) has had a significant impact on healthcare systems globally. Timely and accurate diagnosis is crucial for controlling and preventing outbreaks. Computed tomography (CT) has been demonstrated as an effective diagnostic tool for COVID-19. Deep learning techniques have been applied in related fields to solve relevant problems. Normally, they achieve good performance on large datasets directly. However, due to the need to protect patient privacy and the sudden outbreak of the epidemic, the amount of publicly available CT data is insufficient for utilizing these methods. In addition, some natural images, which are used for model pre-training, are quite different from chest CT images, causing some inevitable errors. To address these issues, a targeted self-supervised attention network based on an improved inception ResNet called SIRANet is proposed. SIRANet incorporates a self-supervised learning method during the pre-training process and introduces a attention module to improve feature selection and focus on key informative features. Moreover, a transfer learning-based strategy for COVID-19 detection is presented and experiments are conducted on three widely used datasets, including ImageNet, an unlabeled chest CT dataset, and a COVID-19 CT dataset. Experimental results show that SIRANet outperforms other state-of-the-art techniques in terms of accuracy, precision, recall, and f1-score, demonstrating its effectiveness in capturing key informative features from unlabeled and labeled images. The implementation of SIRANet is available at https://github.com/Xiiin6/COVID-19.

In recent years, methods based on deep convolutional neural networks have made great progress in the field of image super-resolution reconstruction. However, mainstream approaches generally establish many network layers, leading to high computational costs and memory usage that are unsuitable for resource-limited edge devices. To alleviate this issue, a lightweight inverted residual attention network (IRAN) is proposed to obtain better super-resolution reconstruction performance with lower parameters and computation. The main structure of the IRAN consists of a series of inverted residual attention groups (IRAGs), which are mainly composed of several inverted residual attention blocks (IRABs). IRAB effectively reduces the network parameters and computation while extracting depth features by introducing the inverted residual structure and uses the simple channel attention mechanism to learn the important channel feature information. In addition, an enhanced spatial attention mechanism is introduced at the beginning and end of IRAG to further improve the reconstruction performance of the network. The experimental results show that compared with the mainstream lightweight networks, not only the peak signal-to-noise ratio and structural similarity of image quality metrics are better but also the parameters and the computational effort of the network are lower.

Low-light (LL) images make subsequent computer vision tasks difficult due to its low contrast. To solve this problem, the LL image enhancement problem is regarded as narrowing the gap between LL images and normal-light images in the process of iterative learning. A continuously updated connection network (CUCN) that is inspired by the recent development of convolutional neural networks is proposed. The proposed CUCN is composed of a continuously updated 4-units module (CU4UM), a feature fusion module (FFM), and a color enhancement module (CEM). The CU4UM adopts a connection method to combine four U-shaped structures with continuously updated parameters. To retain the global and local details while brightening an image, the FFM effectively combines the features of different layers in different U-shaped structures and assigns appropriate weights to each channel. The CEM takes the residuals of multiple features of different scales and performs iterative weighting to obtain more natural color information. The proposed method is evaluated on different public datasets, and the results show that the proposed CUCN method is superior to other state-of-the-art methods in terms of both subjective and objective metrics.

In recent years, deep neural networks have exhibited promising performance in the single-image super-resolution (SISR) task by designing complex network structures from the low-resolution (LR) images to the high-resolution (HR) images and learning the mapping function. However, the SISR task is a widely known ill-posed problem. Since it may map one LR image input to multiple HR images, there are always undesired structural distortions in the recovered SR images. We creatively propose a SISR network with dynamic multi-task learning and multi-level mutual feature fusion (DMMFSR), alleviating structural distortion problems while maintaining more edge structure. Specifically, we utilize a DML block (DMLB) to train low-frequency and high-frequency image information extraction networks so the two networks can mutually promote each other. In addition, we also use a multi-level mutual feature fusion block to fuse image and edge features at different levels. Extensive experiments demonstrate the effectiveness of our proposed DMMFSR. Meanwhile, our model achieves comparable peak signal-to-noise ratio and structural similarity scores on multiple SR benchmarks compared with state-of-the-art SR methods.

The integrity of the blade surface is an essential guarantee for the safe operation of wind turbines. It is crucial to research damage detection of blade surface cracks. Based on the improved YOLOv5 model, D-YOLO-v5 is proposed by improving the loss function, objective classification function, and activation function, extending the application of image processing, artificial intelligence, and pattern recognition to investigate intelligent recognition of cracks on the surface of blades. The blade cracks object detection test was completed by LabelImg image annotation, D-YOLO-v5 training parameters adjustment, test parameters selection, prediction model construction, model training, and model prediction steps. The recognition of cracks under different conditions, such as normal light, dark light, and external interference, is subsequently completed. The values indicated that the object detection model parameters of D-YOLO-v5 were neither over-fitted nor under-fitted. In this case study, the accuracy of the blade surface crack-detection model is 98.62%, the mean average precision value is 49.12%, the recall value is 92.20%, and the recognition speed of a single image is 0.01 s. The simulation concludes that the proposed method can meet the accuracy and real-time requirements of practical engineering.

Due to the complex underground environment and the small object of the helmet, the detection accuracy is low when the original YOLOv7 algorithm is used to detect whether the mine personnel wears the helmet, which cannot be applied to the actual operation site. In response to this problem, we proposed an FM-YOLOv7 mine personnel helmet detection. First, to improve the feature extraction ability of the shallow network and enhance the representation ability of the model on the helmet, we propose the fused-MBCA (fused-MBConv with the coordinate attention) module. Second, to improve the detection ability of small objects, enable the fused features to obtain high-level semantic information and low-level details from different scales, and have more extensive receptive fields, we propose the multi-scale feature fusion efficient layer aggregation networks. Finally, to accelerate the convergence of the model and improve the regression accuracy, we use efficient intersection over union as the bounding box regression loss function. These experiments are based on the self-built mine personnel safety helmet dataset. The results show that the FM-YOLOv7 model outperformed the other six algorithms. The mAP@0.5 of the proposed model can reach 85.7%, which is 1.4% higher than the original YOLOv7 model. Also, the improved YOLOv7 model achieves 91 frames per second in detection speed, which detects whether the mine personnel wears a safety helmet in real time.

Growth stage (GS) is an important crop growth metric commonly used in commercial farms. We focus on wheat and barley GS classification based on in-field proximal images using convolutional neural networks (ConvNets). For comparison purposes, use of a conventional machine learning algorithm was also investigated. The research includes extensive data collection of images of wheat and barley crops over a 3-year period. During data collection, videos were recorded during field walks at two camera views: downward looking and 45 deg angled. The resulting dataset contains 110,000 images of wheat and 106,000 of barley taken over 34 and 33 GS classes, respectively. Three methods were investigated as candidate technologies for the problem of GS classification. These methods were: (I) feature extraction and support vector machine, (II) ConvNet with learning from scratch, and (III) ConvNet with transfer learning. The methods were assessed for classification accuracy using test images taken (a) in fields on days imagined in the training data (i.e., seen field-days GS classification) and (b) in fields on days not imagined in the training data (i.e., unseen field-days principal GS classification). Of the three methods investigated, method III achieved the best accuracy for both classification tasks. The model achieved 97.3% and 97.5% GS classification accuracy for seen field-day test data for wheat and barley, respectively. The model also achieved accuracies of 93.5% and 92.2% for the principal GS classification task for wheat and barley, respectively. We provide a number of key research contributions: the collection curation and exposure of a unique GS labeled proximal image dataset of wheat and barley crops, GS classification, and principal GS classification of cereal crops using three different machine learning methods as well as a comprehensive evaluation and comparison of the obtained results.

Due to the increase of crime and terror, security concerns are rising rapidly every day. The use of surveillance cameras for abnormal behavior detection has become an indispensable part of human beings. But the performance of most of the developed systems is not up to the mark because of the low performance and accuracy in detecting the abnormality in the videos due to mainly the presence of noise. The videos captured by the surveillance camera are generally born with no or more noise due to various reasons. To resolve such issues, we provide a snapshot regarding different categories of noise and handcraft techniques to resolve them. Non-local means, block matching, and 3D filtering filters perform astonishingly well while denoising the images. We also present a robust unsupervised deep learning model called deep stacked denoising autoencoder (DSDAE) for denoising the images and further use it for abnormal activity detection and localization in the videos. Our approach has achieved a noteworthy result in image denoising compared to other handcraft-based techniques. DSDAE uses a separate encoder for the extraction of appearance features using clean and noisy images and motion features through the optical flow images. Early fusion is done in the extracted features and passed to the decoder. Only those pixels whose reconstruction error is greater than the threshold will be considered abnormal pixels. Experiment results are compared quantitatively/qualitatively with the recent competitive state-of-the-art methods in the publicly available benchmark datasets Ped1, Ped2, CUHK Avenue, and ShanghaiTech that demonstrate the superior accuracy and performance of our DSDAE. The obtained area under the curve of DSDAE in Ped1, Ped2, CUHK Avenue, and ShanghaiTech is 98.14%, 97.92%, 95.89%, and 96.7%, respectively, whereas equal error rate for the same datasets is 5.4%, 4.5%, 12.03%, and 7.8%, respectively.

Segmentation of images is a critical pre-processing step in image processing. The objective of segmentation is to divide an image into two homogeneous segments based on similar features. Recently, energy-based segmentation methods, such as the level set algorithm, have exhibited remarkable performance in segmenting various types of image data. However, such methods fail to produce satisfactory results on textured images, leading to inaccurate foreground and background segmentation. To address this issue, the proposed method utilizes a feature space based on image local energy, which is independent of image intensity. This feature space provides a more robust representation of texture in the image. By taking advantage of a unique feature space, images can be represented with a sum of Legendre polynomials. The reproduced image is injected into the L2S algorithm. The level set function is optimized from the average difference of Legendre polynomial coefficients to provide the best representation of the image. Consequently, the proposed method results in better segmentation accuracy, with an average increase of 19.33 compared to existing state-of-the-art methods.

Human pose activity recognition (HPAR) offers a wide range of applications due to the widespread use of collection devices such as smartphones and video cameras, as well as its capacity to gather human activity data. Electronic devices and applications continue to evolve, and breakthroughs in artificial intelligence (AI) have transformed the capacity to extract deeply buried information for accurate recognition and interpretation. We propose a systematic design for integrating conventional networks and constraints into the attention framework for learning long-range dependencies, thereby achieving end-to-end pose estimation with flexibility and scalability. The proposed method modifies the temporal receptive field using a multi-scale structure of dilated convolutions and can be adapted to a causal model for real-time performance. Our approach achieves state-of-the-art performance on the task of three-dimensional HPAR and outperforms previous methods while maintaining a lower complexity cost.

Arbitrary style transfer has flexible applicability and aims to learn the painting styles of different artists through one model training and re-render to everyday photos. Existing domain-to-domain style transfer methods based on generative adversarial networks achieve good results in image generation quality but need more flexibility in various style transfer tasks. In the image-to-image style transfer method, focusing on the fusion of content image and style image can learn local detail texture very well. Still, there are some defects in the preservation of the content structure. To alleviate the above two problems, we leverage both advantages, using attention improves style transfer via contrastive learning. Specifically, the attention mechanism is used to fuse the multi-scale features of the content and style images. The previous shallow features are fully utilized to supplement the texture information lost by the deep features. Then the currently learned style image is distinguished from other style images by contrastive learning modules, thereby enhancing the ability of the network to learn style images. We also derive a local loss based on the transferred blocks to improve the visual quality of the generated images. We conduct extensive qualitative and quantitative analyses, the results of which demonstrate the effectiveness of our approach.

Color quantization (cq) is a classical image processing operation that reduces the number of distinct colors in a given image. Although the idea of cq dates back to the early 1970s, the first true cq algorithm, median-cut, was proposed later in 1980. Since then, hundreds of publications have investigated the topic of cq, proposing dozens of algorithms. A vast majority of these publications demonstrate their results on small datasets, containing a handful of images of mixed quality. Furthermore, the reproducibility of cq research is often limited due to the use of private test images or public test images with multiple non-identical copies on the World Wide Web or restrictive licenses. To address these problems, we curated a large, diverse, and high-quality dataset of 24-bit color images called cq100 and released it under a permissive license. We present an overview of cq100 and demonstrate its use in comparing cq algorithms.

We propose a fast image dehazing (FID) algorithm from a reverse perspective. Instead of estimating more accurate transmission or atmospheric light, our work focuses on the residual scene radiance adjustment, which can improve the quality of the dehazed results and address dim effects. First, we formulate an over-dehazed scene and the respective residual scene by pixel-wise dark channel prior. Second, the transmission map is smoothed by a threshold L₁-TV filter, and the over-dehazed result is recovered. Third, the residual scene is adjusted using local color information and Retinex in the brightness domain. Finally, the dehazed scene radiance is created by fusing the over-dehazed result and the adjusted residual scene. FID does not require any training process or complex transmission and atmospheric light estimation, making this algorithm very fast and robust. Experimental results demonstrate that the proposed FID outperforms state-of-the-art dehazing algorithms regarding efficiency and dehazing effect.

Metal gaskets are an essential component of the manufacturing industry, and detecting end-face defects is critical to ensuring their quality. However, current detection methods struggle to detect such defects due to the gaskets’ uneven end-face structure and small defect size. To address such issues, an approach called SR-HorNetYOLO has been proposed. This approach combines traditional machine vision and deep learning techniques. The visual saliency method is used to simplify the image and remove irrelevant features. YOLOv5 is then optimized in three ways: the GhostNet backbone replaces the CSPDarkNet53 backbone to reduce model weight, the self-attention mechanism and bottleneck transformer block improve the precision of detecting small targets, and the feature pyramid is improved through the HorNet network to enhance the predictive performance of the network. Finally, a classification module and position module are employed to classify and locate three-fusion proportion characteristic images. The experimental results show that the SR-HorNetYOLO method achieves a precision of 93.3% and a recall rate of 94.0%.

With the rapid development of multimedia editing technology and DeepFake technology, image integrity and authenticity meet more challenges. Most existing methods only focus on improving the accuracy of tamper detection and localization, but ignore the potential tampering risk, which is related to the saliency. There are uneven potential tamper threats to any graphic images, and it is interesting to exploit saliency to adaptively assign embedding cost. We propose an active forensics scheme for tamper localization by adaptively adjusting cost assignment. The experimental results demonstrate a significant improvement in transparency, localization accuracy, and robustness against unintentional attacks.

Many existing image captioning methods only focus on image objects and their relationships for generating image captions, ignoring the text present in an image. Scene text (ST) contains crucial information to understand an image and facilitating reasoning. The existing methods fail to establish strong correlations between optical character recognition (OCR) tokens, as they have limited OCR representation power. Further, these methods have not efficiently used the positional information of the text. In this work, we have proposed an ST-based image captioning model (Trans-MAtt) based on a multilevel attention mechanism and relation network. We have used relation networks to enhance the connections between ST tokens. We have employed a multi-level attention method, which comprises of spatial, semantic, and appearance attention modules that precisely define the image. To represent context-enriched ST tokens, we use a combination of appearance, location, FastText, and PHOC features. We predict the ST location in the image, which is further integrated with the generated word embeddings for final caption generation. Experiments on the TextCaps dataset demonstrate the effectiveness of the proposed Trans-MAtt model, where it outperforms the current best model by 3.4% on B-4, 2.9% on METEOR, 3.3% on ROUGE-L, 3.1% on CIDEr-D, and 4.1% on SPICE metric scores. Our experiments on the Flickr30k and MSCOCO datasets demonstrated the superiority of our proposed model over existing methods.

Deeplab series semantic segmentation algorithms extract target semantic features using deep layers of a convolutional neural network, resulting in target features lacking detailed information, such as edges and shapes extracted by shallow layers. Deeplabv3plus uses atrous convolution to obtain feature maps, which lose some image information. All of the above have an impact on segmentation performance improvement. In response to these issues, which reduce segmentation performance, we propose an image semantic segmentation algorithm based on a multi-expert system that builds multiple expert models based on the Deeplabv3plus network architecture. For the target image, each expert model makes independent judgments, and the segmentation results are obtained through the ensemble learning of these expert models. Expert model 1 employs the proposed attention-based atrous spatial pyramid pooling (C-ASPP) module to capture richer global semantic information via a parallel attention mechanism and ASSP module. Expert model 2 designs a feature fusion-based decoder that uses a feature fusion approach to obtain detailed information. Expert model 3 introduces a loss function in the Deeplabv3plus network for supervised detailed information loss. The final segmentation results are generated by adjudicating the results derived by the different expert models, which improves the segmentation performance by compensating for the loss of detailed information and enhancing the semantic features. Evaluated on the commonly used semantic segmentation datasets PASCAL VOC 2012 and CamVid, the algorithm’s mIoU reached 82.42% and 69.18%, respectively, which were 2.46% and 1.82% higher than Deeplabv3plus, proving the better segmentation performance of the algorithm.

Densely sampled light fields (LFs) are critical for their further applications, such as digital refocus and depth estimation. However, it is costly and time-consuming to capture them. LF reconstruction, which aims at reconstructing a densely sampled LF from a sparsely sampled one, has attracted extensive attention of researchers. Although existing methods have achieved significant progress, these methods synthesize novel views, either through depth estimation and image warping, which depend heavily on the accuracy of the depth maps and are prone to cause artifacts at occluded regions, or by stacking multi-layer convolutions to learn the inherent structure of the LF, which will result in blurring results due to limited receptive fields when processing scenes with large disparities. We propose a transformer-based neural network for LF reconstruction (termed as LFRTR). Specifically, two novel transformers are introduced, namely angular transformer and spatial transformer. The former can fully explore angular information and correlations among different views, whereas the latter can capture local and non-local spatial texture information within each view. Moreover, dense skip connections are employed to enhance information flow between different layers. Thanks to the inherent global modeling ability of self-attention, the proposed LFRTR can reconstruct high-quality densely sampled LF in complex scenarios, such as large disparity, occlusion, and reflection. Experimental results on both synthetic and real-world LF datasets show that the proposed LFRTR outperforms other state-of-the-art methods in terms of both visual and numerical evaluations.

The scattering and absorption of suspended particles in water severely restrict the quality of underwater imaging. Several works have been proposed to improve the dehazing effect, but their efficiency was not considered. However, the processing efficiency of dehazing is an important indicator of the computing cost in an active application. An improved dehazing method that analyzes and optimizes the estimation accuracy of polarization parameters in the traditional underwater polarization imaging model is proposed. The degree of linear polarization of the target object light and the backscattered light and the transmission coefficient in the scattering medium are jointly estimated by the stochastic gradient descent method. The experimental results indicate that the proposed method can better preserve image details at a faster speed than the traditional underwater dehazing imaging method, especially in high-turbidity water environments. The average speed of the algorithm for restoring a 1024 × 768 pixel image is about 6.2 ms.

Image synthesis is a critical task in various computer vision technologies, and lots of methods tried to translate semantic images into realistic ones for controllable synthesis. With the increasing image resolution, networks are becoming larger, and applications of related methods are restricted. To alleviate the problem, we propose a lightweight mutable network for semantic image synthesis. The network is based on generative adversarial networks. We introduce the feature pyramid architecture to the generator and reduce the hidden node numbers. We also design a mutable scheme where the pyramid will involve fewer layers for smaller images. To improve the performance of the lightweight generator, we further propose a weighted discriminator and a refined loss. The experiments on several public datasets show that our method is effective and achieves competitive performance, proving that a small network can also achieve high-quality semantic image synthesis.

Iris defect texture inpainting is a challenging problem that is not only limited by a lack of research but also by requiring a higher degree of texture refinement than other types of images. In the field of image inpainting, most recent research has focused on designing improved encoder-decoder models. Solving the image blurring problem caused by autoencoder models has become a key factor in judging their merits. Generative adversarial networks were designed based on a hybrid attention generative adversarial network (Hybrid A-GAN) mechanism to repair missing iris textures. The generator is based on the encoder-decoder structure and introduces two attention mechanisms to enable obtaining the correlation between channels in the feature map and pixel importance in space, which enhances the network’s ability to utilize feature information. Moreover, the improved jump connection effectively fuses the high-level features with the low-level features after weighting, which prevents the information loss caused by the downsampling process and enhances the image generation capability. In addition, the joint Wasserstein generative adversarial network-gradient penalty and L1 loss jointly guide network training, which further enhances network generation performance and ensures global consistency of the generated images. Extensive repair experiments and recognition experiments conducted on three publicly available datasets demonstrated that hybrid A-GAN has excellent repair capability and generalization performance.

Cross-domain face matching is known as heterogeneous face recognition (HFR), which is more valuable than traditional face recognition and plays a key role in public safety. It remains a challenging problem due to insufficient heterogeneous data and large domain discrepancies. Recent work has focused on learning common features from different visual domains and synthesizing face images from other domains into the visible domain to reduce domain discrepancy. The former performs domain transformation during face recognition, which requires additional runtime. The latter has poorer performance limited by insufficient heterogeneous data. We propose a one-way multimodal image-to-image translation (OMIT) to tackle the problem of insufficient heterogeneous data. Specifically, we learn the generation of heterogeneous data from small-scale paired heterogeneous training data, and an identity-preserving loss is imposed on the generated heterogeneous images to ensure their identity consistency. Then, we transform large-scale visible data of abundant identity information into paired heterogeneous data, which addresses the lack of identity diversity due to small-scale heterogeneous data. The generated pairwise heterogeneous data can be directly used to train the HFR network to improve recognition accuracy and precision. Finally, our method exhibits generated quality improvements on BUAA, CASIA NIR–VIS 2.0, Oulu NIR–VIS, and Tufts face database. Heterogeneous datasets generated by OMIT improve the performance of HFR networks.

Oriented bounding box regression is crucial for oriented object detection. However, regression-based methods often suffer from boundary problems and the inconsistency between loss and evaluation metrics. A modulated Kalman intersection over union (IoU) loss of approximate SkewIoU is proposed, named MKIoU. To avoid boundary problems, we convert the oriented bounding box to Gaussian distribution then use the Kalman filter to approximate the intersection area. However, there exists significant difference between the calculated and actual intersection areas. Thus, we propose a modulation factor to adjust the sensitivity of angle deviation and width-height offset to loss variation, making the loss more consistent with the evaluation metric. Furthermore, the Gaussian modeling method avoids the boundary problem but causes the angle confusion of square objects simultaneously. Thus, the Gaussian angle loss (GA loss) is presented to solve this problem by adding a corrected loss for square targets. The proposed GA loss can be easily extended to other Gaussian-based methods. Experiments on three publicly available aerial image datasets, DOTA, UCAS-AOD, and HRSC2016, show the effectiveness of the proposed method.

Recognition of individuals using periocular images has become popular due to its ease of use in situations in which it is difficult to capture full-face regions or high-resolution iris images. Practical surveillance systems recognize individuals from images acquired in various imaging conditions. Recognition of individuals in cross-spectral environments is of practical interest as it can be used to match probe and gallery images that are captured in distinct wavelength domains. The process of matching probe near-infrared images with a gallery of visible images is significantly challenging due to variations in appearance and illumination conditions. In periocular regions, alleviating this wide appearance gap from such local regions is highly challenging. In this work, we propose a dual-spectrum network that considers both high-level deep features and low-level handcrafted attributes to match periocular images in cross-spectral scenarios. A mapping network is designed to establish the relationship between both types of features and generate mapped features that carry discriminative information relevant to both feature domains. The mapped features are passed through a distance layer and subsequent classification block to determine whether they are a genuine pair or an imposter pair. The network is evaluated on a publicly available PolyU dataset as well as two newly created in-house CASIA and IITBBS cross-spectral periocular datasets. The IITBBS dataset consists of visible and near-infrared periocular images from 100 subjects captured in unconstrained imaging conditions with pose and accessory variations. Extensive experiments indicate that the proposed network achieves verification accuracies of 97.96%, 96.48%, and 97.89% on the PolyU, CASIA, and IITBBS datasets, respectively. Performance analysis with existing methods confirms that our network achieves comparable results on the PolyU and CASIA datasets. In addition, the network outperforms existing works on the challenging IITBBS dataset by achieving improvements of 0.43% and 0.16% in terms of verification accuracy and genuine acceptance rate at a 0.1 false acceptance rate.

The purpose of image steganalysis is to determine whether the carrier image contains hidden information or not. Since JPEG is the most commonly used image format over social networks, steganalysis in JPEG images is also the most urgently needed to be explored. However, in order to detect whether secret information is hidden within JPEG images, the majority of existing algorithms are designed in conjunction with the popular computer vision related networks without considering the key characteristics of image steganalysis. It is crucial that the steganographic signal, as an extremely weak signal, can be enhanced during its representation learning process. Motivated by this insight, we introduce a representation learning algorithm for JPEG steganalysis that is mainly consisting of a graph attention learning (GAL) module and a feature enhancement module. The GAL module is designed to avoid global feature loss caused by the local feature learning of convolutional neural network and reliance on depth stacking to extend the perceptual domain. The feature enhancement module is applied to prevent the stacking of convolutional layers from weakening the steganographic information. In addition, pretraining as a way to initialize the network weights with a large-scale dataset is utilized to enhance the ability of the network to extract discriminative features. We advocate pretraining with ALASKA2 for the model trained with BOSSBase + BOWS2. The experimental results indicate that the proposed algorithm outperforms previous arts in terms of detection accuracy, which has verified the superiority and applicability of the proposed work.

Accurate identification of cracks is of great significance for maintaining the health of the equipment. However, the low saliency of cracks in some composite or metal surfaces affects the detection accuracy of object detection algorithms. For example, small cracks on the inner surface of wind turbine blade (WTB) may be similar in color to the substrate or face complex background textures. Taking WTB cracks as low saliency crack samples, we propose a multimodal object detection convolutional neural network that fuses infrared images with visible images to detect cracks more accurately. The proposed network contains the CenterNet network with an existing fast and efficient mid-level fusion structure. First, we optimized the fusion structure to make it more suitable for extracting crack features. To address the problem that severe background interference in multimodal images affects the detection performance, we add channel attention to the fusion structure and train the improved network using a stepwise training method to enhance the framework’s ability to filter background interference information. Finally, the effectiveness of the improvements was verified by ablation experiments and feature map analysis, and the phenomena of wrong detection, missed detection, and repeated detection were reduced. The evaluation results show that the proposed multimodal object detection network is able to detect the low saliency WTB cracks more effectively, and the improvement of the network also results in a 6.22% increase in average precision. In addition, this method can be extended to other materials or scenes to identify very inconspicuous objects, replacing manual inspection in more challenging defect detection tasks.

To enhance the working efficiency of apple picking systems by improving the accuracy of object detection in natural scenes, an improved apple detection network based on YOLOX-s is proposed. The self-attention residual module is added to the last layer of the improved YOLOX-s, which is used to add global feature information for small apples. An additional object detection head is added in YOLOX-s to strengthen the feature information of dense objects. Extra convolutional branches are added into the spatial pyramid pooling structure to enhance feature fusion and compensate for lost object location information. The loss function of the network is replaced by the α-CIoU loss, which is used to improve the bbox regression accuracy by up-weighting the loss and gradient of the high IoU prediction box. Experiment results show that the mAP₅₀ value and the recall rate of the improved network reached 91.8% and 97%, respectively. Therefore, the improved network is superior to the existing detection networks in detection accuracy. Meanwhile, the detection time increased slightly, but it still meets the requirements of real-time detection of the picking system.

Compared with training multiple single-scale models to achieve different scale image super-resolution (SR), using the internal correlation of multiple scales to aggregate multi-scale learning into a single model can effectively reduce the network parameters and improve efficiency. Correspondingly, it is important to study how to take the advantage of inter-scale correlation at a fixed model size for the performance improvement of multi-scale SR. We combine the capture of richer context dependencies and convolutional response modulation to improve the discriminative ability of the model, and then propose a global context-aware feature modulation network, which has a stronger representational capability for multi-scale SR. Specifically, we first model the global context based on the self-attention mechanism, which can effectively extract rich global information from convolutional features. Then, we propose to modulate the feature responses using global contextual dependencies in spatial and channel dimensions respectively. Further, we construct the global context-aware feature modulation block, which can be stacked to form a deep feature modulation network and applied to the unified multi-scale SR architecture to learn inter-scale correlation. Extensive experiments demonstrate that the proposed global context-aware modulation is effective, which can improve the performance of multi-scale SR task both quantitatively and qualitatively.

Herein, a lightweight “You Only Look Once” (YOLOv5) ultrahigh voltage (UHV) transmission line insulator self-detonation detection algorithm based on the attention mechanism and Ghostnet is proposed to address the multi-scale features of UHV insulator aerial images. Small-scale defective pieces cannot be detected accurately by conventional algorithms in a complex power system inspection environment. To suppress the complex background interference, a YOLOv5-based spatial and channel convolution attention model is used to enhance the saliency of the target to be detected by weighting the feature layers and feature maps. The convolutional structure of the original cross stage partial (CSP_X) structure is replaced by the Ghostnet network to generate more similar feature maps by performing linear operations on the redundant feature maps so that more feature maps can be generated using fewer parameters to achieve a lightweight network and reduce the network parameters. To address the problem of missed and false detection caused by inadequate expression of the target features to be detected, the original neck feature pyramid network (FPN) + path aggregation network (PAN) structure is modified to a bidirectional FPN structure such that the target multi-scale features are effectively fused. Finally, SCYLLA-IoU is used as the loss function to accelerate the model convergence and improve the detection accuracy, and migration learning parameter sharing combined with the model training strategy of freeze and thaw training is performed for non-generalization of the network owing to small sample datasets. The effectiveness of the proposed algorithm is verified by training it with data obtained from UHV patrols. Experimental results show that the proposed algorithm accurately monitors insulator self-detonation targets in complex environments with an average detection accuracy of 97.4% and a significant reduction in the size of models and parameters, which has good practical value.

Face recognition, being a widely utilized and profoundly challenging task within the field of computer vision, has witnessed the emergence of numerous practical methodologies in recent years. Among these, dictionary learning-based methods have garnered extensive attention and have found widespread application. Nevertheless, it is worth noting that the majority of dictionary learning methods suffer from certain limitations. On the one hand, their primary focus lies solely on the resolution of the original images, thereby rendering them susceptible to the impact of resolution changes encountered in real-world scenarios. On the other hand, their ability to acquire a robust dictionary is hindered by the inadequacy of available training samples. We propose a multi-resolution dictionary learning method based on sample extension and label embedding. We generate virtual samples for the original training samples and convert all samples into different resolutions. In addition, noise constraints are applied to the virtual samples to enhance robustness. Then, we construct the label embedding term using the label information of the atoms and embed it on a multi-resolution dictionary learning model for training. The recognition rates of the method on the ORL, extended Yale B, CMU PIE, and GT datasets are 93.72%, 92.57%, 97.79%, and 73.91%, respectively.

Accurately segmenting industrial smoke in videos plays a crucial role in assessing pollution levels based on smoke image evaluation. However, existing fully convolutional networks (FCNs) face challenges in precisely segmenting the edges of industrial smoke and exhibit low extraction and segmentation accuracy for small target smoke. To address this issue, we propose a video segmentation method specifically designed for industrial smoke. This method utilizes the dynamic FCN-Gaussian mixture model (GMM) along with a multi-scale fusion module and an attention module. The FCN-GMM effectively extracts dynamic feature information from spatiotemporal data, capturing motion in video or image sequences while preserving spatial details. The key innovation of FCN-GMM lies in integrating dynamic and static networks through a neural network, enabling the capture of features in both the temporal and spatial domains. Our approach begins by constructing a dynamic feature extraction network that captures spatial and temporal feature information separately during the training process, thereby enhancing the extraction of smoke edges. Additionally, we introduce a mechanism for multi-scale feature fusion and an attention module to effectively extract information related to small target smoke. Our experimental results demonstrate that our network accurately segments significant target smoke compared with FCNs. Furthermore, the network prioritizes the consideration of smoke edge information and improves the extraction of small target smoke, thereby enhancing the overall accuracy of smoke image segmentation with an increase of up to 10% in the intersection over union index.

Agriculture is a crucial sector for every country’s economy, as it provides livestock and crops that are essential for farmers and their families’ survival. Crop diseases significantly impact farmers’ livelihood, making early detection and accurate diagnosis essential for informed crop management decisions that can improve yield and crop quality. While the manual diagnosis of plant diseases has limitations, research on automatic plant disease identification and categorization using computer vision algorithms is a popular research topic. We focus on leaf disease recognition and compare classification techniques to object detection techniques. Classification techniques in leaf disease recognition involve assigning an input image of a leaf to one of the pre-defined classes or categories, such as “healthy” or “diseased.” Object detection techniques, on the other hand, involve identifying the presence of an object of interest (e.g., a leaf) in an image and locating its boundaries, in addition to classifying the object. The advantage of object detection techniques over classification techniques in leaf disease recognition is that they provide additional information about the location of the disease on the leaf, which can be useful for tasks such as disease diagnosis and monitoring. In our work, 5100 chilli leaf images of 6 different classes were collected and labeled using the makeSense artificial intelligence (AI) annotating tool; we applied the YOLOv5s object detection model, producing a precision of 0.951, recall of 0.926, mAP@0.5 of 0.959, mAP@0.5:0.95 of 0.826, and accuracy of 95%. We also proposed an enhanced single-stage object detection model that employs the exponential linear unit activation function in the convolution layers and the sigmoid activation function at the output layer, resulting in a precision of 0.998, recall of 0.998, mAP@0.5 of 0.995, mAP@0.5:0.95 of 0.975, and accuracy of 99.83% for the chilli leaf dataset.