Detecting small objects in remote sensing images is a challenging task. Existing object detectors for remote sensing images suffer from two issues: (1) insufficient feature extraction for small objects in the backbone network and (2) feature misalignment and information loss for small objects in the neck network, leading to poor detection performance on small objects. To address these challenges, a small object detector named CABDet for remote sensing images that combines context and attention mechanisms is proposed. Specifically, an enhanced ResNet50 is designed as a novel backbone network that adaptively adjusts the size of receptive fields to fully extract feature information of small objects. Additionally, an adaptive multiscale feature pyramid network (AM-FPN) is proposed. To alleviate the problem of feature misalignment for small objects, AM-FPN leverages self-attention mechanisms to establish semantic and spatial dependencies between adjacent feature layers. Then to mitigate the issue of information loss for small objects, AM-FPN captures semantic dependencies between subregions of current layer features through self-attention mechanisms to preserve channel information. Extensive experiments were conducted on two demanding remote sensing datasets, namely dataset for object detection in aerial images and UCAS-high resolution aerial object detection dataset, to demonstrate the effectiveness of the proposed methodology in achieving superior detection performance when compared with contemporary state-of-the-art approaches.