A weight recognition scheme is proposed based on the analysis of plastic optical fiber (POF) specklegrams. A simple experiment is performed for data acquisition comprising specklegram images corresponding to different external applied weights ranging from 0 to 3 kg in steps of 0.5 kg on the POF. These specklegram images are further split into training, validation, and test datasets for employment in the convolutional neural network (CNN). The model is trained and validated with a sufficient number of epochs (iterations) to obtain optimal training and validation accuracies (>90 % ); it is then used for recognizing (classifying) the unseen test dataset images. A user-defined CNN model is optimized using four different optimizers: Adam, AdaMax, Nadam, and RMSProp. The recognition (or test) accuracy of these optimizers is compared. The Nadam optimizer has the highest recognition accuracy of 93.1% for increasing weights and 91.9% for decreasing weights. Furthermore, we investigate the impact of two parameters, namely temperature effect and information loss of specklegrams, on the model’s weight recognition accuracy. The temperature effect is studied for finite temperatures ranging from 29°C to 35°C with 3°C step fluctuations around the ambient temperature of 25°C. We find that the maximum deviation in recognition accuracy is about 1.2%. To quantify the information content (speckles) in each blocked specklegram, Shannon entropy (SE) is estimated. These images are then used in our existing CNN model with the Nadam optimizer to evaluate recognition performance with blocking. With increasing blocking, we observe a decrease in SE and recognition accuracy. This analysis suggests that, even with information loss in the specklegrams at ≤20 % blocking, a good representation of weights with high recognition accuracy (>80 % ) can be obtained. The presence of a small physical obstacle in front of the imaging system can cause information loss in specklegrams to manifest in practical scenarios.