Label-Free Visual Concept Drift Detection via Classifier Two-Sample Tests and Characteristic Function Embeddings

Authors

DOI:

https://doi.org/10.31181/jscda41202686

Keywords:

Concept drift, Classifier two-sample test, Semantic shift, Deep representations, Label-free monitoring

Abstract

Deploying deep neural networks in non-stationary environments exposes them to concept drift, silently degrading predictive performance over time. Traditional statistical two-sample tests and distance-based heuristics frequently fail to detect natural, semantic shifts in high-dimensional visual streams. To address this critical blind spot, we propose a real-time, label-free concept drift detection framework that couples the Classifier Two-Sample Test (C2ST) with a novel Characteristic Function Embedding (CFE) bottleneck. Operating directly on deep convolutional representations, this architecture compresses features to resolve dimensionality constraints and ensure highly efficient downstream evaluation. We systematically benchmark our sliding-window framework against classical and modern baselines, including Maximum Mean Discrepancy (MMD), Kolmogorov-Smirnov (KS), and DriftLens, under complex, real-world shifts using the WILDS Waterbirds subpopulation dataset and the PACS domain generalisation benchmark. Our empirical findings demonstrate that distance-based baselines severely degrade under semantic shift; notably, the KS test collapses to a 0.45 score on the Waterbirds dataset, while MMD shrinks by an order of magnitude. In contrast, the proposed C2ST framework retains highly robust discriminative power, achieving above 0.97 AUROC on semantic shifts and saturating effectively on domain shifts. To complement this, a micro-sensitivity analysis on the CIFAR-10 dataset demonstrates that our framework successfully isolates ultra-low intensity synthetic Gaussian drift, maintaining detection capabilities where traditional tests fail. Ultimately, the framework maintains a calibrated false-positive rate on stable, in-distribution streams. These results confirm that the C2ST-CFE architecture provides a mathematically grounded and highly responsive mechanism for safe machine learning deployment in real-world visual applications.

Downloads

Download data is not yet available.

References

Lu, J., Liu, A., Dong, F., Gu, F., Gama, J., & Zhang, G. (2018). Learning under concept drift: A review. IEEE Transactions on Knowledge and Data Engineering, 31(12), 2346–2363. https://doi.org/10.1109/TKDE.2018.2876857

Hovakimyan, G., & Bravo, J. M. (2024). Evolving strategies in machine learning: A systematic review of concept drift detection. Information, 15(12), 786. https://doi.org/10.3390/info15120786

Marathe, S., Nambi, A., Swaminathan, M., & Sutaria, R. (2021). Currentsense: A novel approach for fault and drift detection in environmental IoT sensors. In Proceedings of the International Conference on Internet-of-Things Design and Implementation (pp. 93–105). https://doi.org/10.1145/3450268.345353

Goncalves, V. P., Silva, L. P., Nunes, F. L., Ferreira, J. E., & Araújo, L. V. (2024). Concept drift adaptation in video surveillance: A systematic review. Multimedia Tools and Applications, 83(4), 9997–10037. https://doi.org/10.1007/s11042-023-15855-3

Gama, J., Medas, P., Castillo, G., & Rodrigues, P. (2004). Learning with drift detection. In Brazilian Symposium on Artificial Intelligence (pp. 286–295). https://doi.org/10.1007/978-3-540-28645-5_29

Baena-García, M., del Campo-Ávila, J., Fidalgo, R., Bifet, A., Gavalda, R., & Morales-Bueno, R. (2006). Early drift detection method. In Fourth International Workshop on Knowledge Discovery from Data Streams (Vol. 6, pp. 77–86). https://www.researchgate.net/publication/245999704_Early_Drift_Detection_Method

Ren, S., Zhu, W., Liao, B., Li, Z., Wang, P., Li, K., Chen, M., & Li, Z. (2019). Selection-based resampling ensemble algorithm for nonstationary imbalanced stream data learning. Knowledge-Based Systems, 163, 705–722. https://doi.org/10.1016/j.knosys.2018.09.032

Zheng, X., Li, P., Hu, X., & Yu, K. (2021). Semi-supervised classification on data streams with recurring concept drift and concept evolution. Knowledge-Based Systems, 215, 106749. https://doi.org/10.1016/j.knosys.2021.106749

Idrees, M. M., Minku, L. L., Stahl, F., & Badii, A. (2020). A heterogeneous online learning ensemble for non-stationary environments. Knowledge-Based Systems, 188, 104983. https://doi.org/10.1016/j.knosys.2019.104983

Krawczyk, B. (2017). Active and adaptive ensemble learning for online activity recognition from data streams. Knowledge-Based Systems, 138, 69–78. https://doi.org/10.1016/j.knosys.2017.09.032

Gretton, A., Borgwardt, K. M., Rasch, M. J., Schölkopf, B., & Smola, A. (2012). A kernel two-sample test. The Journal of Machine Learning Research, 13(1), 723–773. http://jmlr.org/papers/v13/gretton12a.html

Greco, S., Vacchetti, B., Apiletti, D., & Cerquitelli, T. (2025). Unsupervised concept drift detection from deep learning representations in real-time. IEEE Transactions on Knowledge and Data Engineering. Advance online publication. https://doi.org/10.1109/TKDE.2025.3593123

Ross, G. J., Adams, N. M., Tasoulis, D. K., & Hand, D. J. (2012). Exponentially weighted moving average charts for detecting concept drift. Pattern Recognition Letters, 33(2), 191–198. https://doi.org/10.1016/j.patrec.2011.08.019

Tran, Q.-T., Le-Khac, N.-A., & Bertolotto, M. (2026). Concept drift detection in image data stream: A survey on current literature, limitations and future directions. Artificial Intelligence Review, 59(1), 33. https://doi.org/10.1007/s10462-025-11428-y

Angelov, P. P., Gu, X., & Principe, J. C. (2017). Autonomous learning multimodel systems from data streams. IEEE Transactions on Fuzzy Systems, 26(4), 2213–2224. https://doi.org/10.1109/TFUZZ.2017.2769039

Basci, P., Greco, S., Manigrasso, F., Cerquitelli, T., Morra, L., et al. (2025). Explaining concept drift via neuro-symbolic rules. In CEUR Workshop Proceedings (Vol. 4132). CEUR. https://ceur-ws.org/Vol-4132/short54.pdf

Lopez-Paz, D., & Oquab, M. (2016). Revisiting classifier two-sample tests. arXiv preprint arXiv:1610.06545.

Koh, P. W., Sagawa, S., Marklund, H., Xie, S. M., Zhang, M., et al. (2021). WILDS: A benchmark of in-the-wild distribution shifts. In Proceedings of the 38th International Conference on Machine Learning (Vol. 139, pp. 5637–5664).

Yu, S., Wu, P., Liang, P. P., Salakhutdinov, R., & Morency, L.-P. (2022). Pacs: A dataset for physical audiovisual commonsense reasoning. In European Conference on Computer Vision (pp. 292–309). https://doi.org/10.1007/978-3-031-19836-6_17

Francis, N., & Ali, A. (2024). Identifying and managing concept drift in machine learning through page-hinkley test: Approaches, obstacles, and resolutions. In International Conference on Recent Trends in Computing (pp. 33–46). https://doi.org/10.1007/978-981-97-8946-7_3

Fellicious, C., Wendlinger, L., & Granitzer, M. (2022). Neural network based drift detection. In International Conference on Machine Learning, Optimization, and Data Science (pp. 370–383). https://doi.org/10.1007/978-3-031-25599-1_28

Muandet, K., Fukumizu, K., Sriperumbudur, B., & Schölkopf, B. (2017). Kernel mean embedding of distributions: A review and beyond. Foundations and Trends in Machine Learning, 10(1-2), 1–141. https://doi.org/10.1561/2200000060

Lesort, T., Lomonaco, V., Stoian, A., Maltoni, D., Filliat, D., & Díaz-Rodríguez, N. (2020). Continual learning for robotics: Definition, framework, learning strategies, opportunities and challenges. Information Fusion, 58, 52–68. https://doi.org/10.1016/j.inffus.2019.12.004

Aljundi, R., Belilovsky, E., Tuytelaars, T., Charlin, L., Caccia, M., Lin, M., & Page-Caccia, L. (2019). Online continual learning with maximal interfered retrieval. In Advances in Neural Information Processing Systems (Vol. 32).

Barddal, J. P., Gomes, H. M., & Enembreck, F. (2015). A survey on feature drift adaptation. In 2015 IEEE 27th International Conference on Tools with Artificial Intelligence (ICTAI) (pp. 1053–1060). https://doi.org/10.1109/ICTAI.2015.150

Krizhevsky, A., & Hinton, G. (2009). Learning multiple layers of features from tiny images (Technical report). University of Toronto. https://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf

Yang, J., Shi, R., & Ni, B. (2021). MedMNIST classification decathlon: A lightweight AutoML benchmark for medical image analysis. In 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI) (pp. 191–195). https://doi.org/10.1109/ISBI48211.2021.9434062

Waller, L. A., Turnbull, B. W., & Hardin, J. M. (1995). Obtaining distribution functions by numerical inversion of characteristic functions with applications. The American Statistician, 49(4), 346–350. https://doi.org/10.2307/2684571

Downloads

Published

2026-08-15

Issue

Vol. 4 No. 1 (2026): Journal of Soft Computing and Decision Analytics

Section

Articles

License

Copyright (c) 2026 Gurgen Hovakimyan, Jorge Miguel Bravo (Author)

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

How to Cite

Hovakimyan, G., & Miguel Bravo, J. (2026). Label-Free Visual Concept Drift Detection via Classifier Two-Sample Tests and Characteristic Function Embeddings. Journal of Soft Computing and Decision Analytics, 4(1), 135-155. https://doi.org/10.31181/jscda41202686