Semi-Supervised Learning with Lightweight Image Classifiers

This study explores the application of the FixMatch algorithm [1] to lightweight convolutional neural networks (CNNs) for semi-supervised image classification in resource-constrained environments.

The FixMatch algorithm is a semi-supervised learning method that combines consistency regularization and pseudo-labeling to make effective use of unlabeled data. It operates by first applying weak augmentations (e.g., random horizontal flips) to both labeled and unlabeled images. For unlabeled samples, the model generates predictions on the weakly augmented versions, and if the confidence of the prediction surpasses a set threshold (commonly 0.95), the predicted label is treated as a pseudo-label. The same unlabeled image is then passed through a strong augmentation pipeline (e.g., RandAugment), and the model is trained to produce consistent predictions between the weak and strong versions of the image. The overall training loss is composed of a supervised cross-entropy loss on labeled data and an unsupervised consistency loss on pseudo-labeled data, with a weighting parameter λu controlling the influence of the unsupervised component. This approach enables the model to learn robust features from a small amount of labeled data while leveraging the structure in the unlabeled data, making it particularly suitable for scenarios where labeled data is limited but unlabeled data is abundant.

The research evaluates three lightweight image classification models including ResNet-8, ResNet-20, and a custom OptVGGNet on the CIFAR-10 dataset using varying proportions of labeled data (0.5%, 2%, and 8%). Hyperparameters such as λu, batch size, learning rate, and confidence threshold were tuned to improve performance. Among the models, OptVGGNet achieved the highest test accuracy of 89.8% with only 8% labeled data. Results showed that higher λu values and smaller batch sizes led to better learning outcomes, while performance dropped significantly with extremely sparse labeled data. t-SNE visualizations further confirmed that feature clustering improved with more labeled examples.