The integration of computer vision into real-world applications is often hindered by the high computational demands of deep learning models and the associated cost of hardware. This paper presents a novel, comprehensive solution for the real-time digitization of a physical chessboard, specifically designed for low-cost embedded platforms. We introduce a complete system that combines efficient classical computer vision algorithms for chessboard localization with a custom-designed, lightweight Convolutional Neural Network (CNN) for piece recognition. The core of our innovation lies in the use of a low-cost, open-source RISC-V platform, which is augmented with a custom, tightly coupled hardware accelerator to offload the most computationally intensive tasks. Our system demonstrates a chessboard localization accuracy of 99.5% and a piece recognition accuracy of 98.2%. Critically, it achieves an average processing latency of 150 milliseconds per frame, enabling true real-time operation on hardware that is an order of magnitude more affordable and power-efficient than conventional GPU-based systems. This work validates a crucial hardware-software co-design paradigm for deploying complex AI tasks on resource-constrained devices, paving the way for a new generation of affordable, intelligent edge applications.

Computer Vision, Chessboard, Deep Learning, Embedded Systems

Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, Nov 1998. [Online]. Available: https://doi.org/10.1109/5.726791

A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in neural information processing systems, 2012, pp. 1097–1105. [Online]. Available: https://doi.org/10.1145/3065386

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778. [Online]. Available: https://doi.org/10.1109/CVPR.2016.90

V. Sze, Y.-H. Chen, T.-J. Yang, and J. S. Emer, “Efficient processing of deep neural networks: A tutorial and survey,” Proceedings of the IEEE, vol. , no. 12, p. 2295–2329, 2017. [Online]. Available: http://doi.org/10.1109/JPROC.2017.2761740

M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “Mobilenetv2: Inverted residuals and linear bottlenecks,” 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018. [Online]. Available: https://doi.org/10.1109/CVPR.00474

F. Chollet, “Xception: Deep learning with depthwise separable convolutions,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1800–1807. [Online]. Available: https://doi.org/10.1109/CVPR.2017.195

B. Zoph, V. Vasudevan, J. Shlens, and Q. V. Le, “Learning transferable architectures for scalable image recognition,” 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018. [Online]. Available: https://doi.org/10.1109/cvpr.2018.

Z. Li, F. Liu, W. Yang, S. Peng, and J. Zhou, “A survey of convolutional neural networks: Analysis, applications, and prospects,” IEEE Transactions on Neural Networks and Learning Systems, vol. 33, no. 12, pp. 6999–7019, 2022. [Online]. Available: https://doi.org/10.1109/TNNLS.2021.3084827

M. A. Czyzewski, A. Laskowski, and S. Wasik, “Chessboard and chess piece recognition with the support of neural networks,” Foundations of Computing and Decision Sciences, vol. 45, no. 4, pp. 257–280, 2020. [Online]. Available: https://doi.org/10.2478/fcds-2020-0014

A. de Sá Delgado Neto and R. Mendes Campello, “Chess position identification using pieces classification based on synthetic images generation and deep neural network fine-tuning,” in 21st Symposium on Virtual and Augmented Reality (SVR), 2019, pp. 152–160. [Online]. Available: https://doi.org/10.1109/SVR.2019.00038

J. Ding, “Chessvision: Chess board and piece recognition,” Stanford University, Tech. Rep., https://web.stanford.edu/class/cs231a/prev_projects_2016/CS_231A_Final_Report.pdf, Accessed: 28/09/2023.

K. Asanovic et al., “The rocket chip generator. eecs department,” University of California, Berkeley, Tech. Rep. UCB/EECS--17, vol. 4, 2016. [Online]. Available: https://github.com/chipsalliance/rocket-chip/tree/47f7b7144727f0340d511d35b9f6c7a91b2a276f

Z. Jerry, B. Korpan, A. Gonzalez, and K. Asanovic, “Sonicboom: The 3rd generation berkeley out-of-order machine,” in Proceedings of the 4th Workshop on Computer Architecture Research with RISC-V (CARRV), pp. 1–7.

Y. Zhou, X. Jin, T. Xiang, and D. Zha, “Enhancing energy efficiency of risc-v processor-based embedded graphics systems through frame buffer compression,” Microprocess. Microsystems, vol. 77, p. 103140, 2020. [Online]. Available: https://doi.org/10.1016/j.micpro.103140

H. Genc et al., “Gemmini: Enabling systematic deep-learning architecture evaluation via full-stack integration,” in Proceedings of the 58th Annual Design Automation Conference (DAC), 2021, pp. –774. [Online]. Available: https://doi.org/10.1109/DAC18074.2021.9586216

C. Matuszek et al., “Gambit: An autonomous chess-playing robotic system,” in IEEE International Conference on Robotics and Automation, 2011, pp. –4297. [Online]. Available: https://doi.org/10.1109/ICRA.2011.5980528

A. Chen and K. Wang, “Robust computer vision chess analysis and interaction with a humanoid robot,” Computers, vol. 8, p. 14, 02 2019. [Online]. Available: https://doi.org/10.3390/computers8010014

P. Kolosowski, A. Wolniakowski, and K. Miatliuk, “Collaborative robot system for playing chess,” in 2020 International Conference Mechatronic Systems and Materials (MSM), 2020, pp. 1–6. [Online]. Available: https://doi.org/10.1109/MSM49833.2020.9202398

B. Tan, “Towards a vision-based mobile manipulator for autonomous chess gameplay,” Master of Science in Technology Thesis, University of Turku. Department of Computing, Faculty of Technology, Robotics and Autonomous Systems, 2023.

E. Civik and U. Yuzgec, “Real-time driver fatigue detection system with deep learning on a low-cost embedded system,” Microprocessors and Microsystems, vol. 99, p. 104851, 2023. [Online]. Available: https://doi.org/10.1016/j.micpro.2023.104851

J. Mas, T. Panadero, G. Botella, A. A. Del Barrio, and C. García, “CNN inference acceleration using low-power devices for human monitoring and security scenarios,” Computers & Electrical Engineering, vol. 88, p. 106859, 2020. [Online]. Available: https://doi.org/10.1016/j.compeleceng.2020.106859

Q. Gui, G. Wang, L. Wang, J. Cheng, and H. Fang, “Road surface state recognition using deep convolution network on the low-power-consumption embedded device,” Microprocessors and Microsystems, vol. 96, p. 104740, 2023. [Online]. Available: https://doi.org/10.1016/j.micpro.2022.104740

A. de la Escalera and J. Armingol, “Automatic chessboard detection for intrinsic and extrinsic camera parameter calibration,” Sensors (Basel, Switzerland), vol. 10, pp. 2027–44, 03 2010. [Online]. Available: https://doi.org/10.3390/s100302027

F. Gao, T. Huang, J. Wang, J. Sun, A. Hussain, and E. Yang, “Dual-branch deep convolution neural network for polarimetric sar image classification,” Applied Sciences, vol. 7, p. 447, 04 2017. [Online]. Available: https://doi.org/10.3390/app7050447

A. J. Bency, H. Kwon, H. Lee, S. Karthikeyan, and B. S. Manjunath, “Weakly supervised localization using deep feature maps,” in European Conference on Computer Vision, 2016, pp. 714–731. [Online]. Available: https://doi.org/10.48550/arXiv.1603.00489

D. Lowe, “Distinctive image features from scale-invariant keypoints,” International Journal of Computer Vision, vol. 60, pp. 91–110. [Online]. Available: https://doi.org/10.1023/B:VISI.0000029664.99615.94

Y. Xie, G. Tang, and W. Hoff, “Chess piece recognition using oriented chamfer matching with a comparison to cnn,” in IEEE Winter Conference on Applications of Computer Vision (WACV), 2018, pp. 2001–2009. [Online]. Available: https://doi.org/10.1109/WACV.00221

S. J. Edwards, “Portable game notation specification and implementation guide: Forsyth-edwards notation,”

W. Rawat and Z. Wang, “Deep convolutional neural networks for image classification: A comprehensive review,” Neural Computation, vol. 29, no. 9, pp. 2352–2449, 2017. [Online]. Available: https://doi.org/10.1162/neco_a_00990

F. Chollet et al., “Keras,” https://keras.io, 2015.

M. Abadi et al., “TensorFlow: Large-scale machine learning on heterogeneous systems,” 2015, software available from tensorflow.org. [Online]. Available: https://www.tensorflow.org/

G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. [Online]. Available: https://doi.org/10.1109/cvpr.2017.243

F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, and K. Keutzer, “Squeezenet: Alexnet-level accuracy with 50x fewer parameters and <0.5mb model size,” 2016. [Online]. Available: https://doi.org/10.48550/arXiv.1602.07360

J. Setoain, M. Prieto, C. Tenllado, A. Plaza, and F. Tirado, “Parallel morphological endmember extraction using commodity graphics hardware,” IEEE Geoscience and Remote Sensing Letters, vol. 4, no. 3, pp. 441–445, 2007. [Online]. Available: https://doi.org/10.1109/LGRS.2007.897398

J. L. Bentley and T. Ottmann, “Algorithms for reporting and counting geometric intersections,” IEEE Transactions on Computers, vol. C-28, no. 9, pp. 643–647, 1979. [Online]. Available: https://doi.org/10.1109/TC.1979.1675432

A. Dörflinger, M. Albers, B. Kleinbeck, Y. Guan, H. Michalik, R. Klink, C. Blochwitz, A. Nechi, and M. Berekovic, “A comparative survey of open-source application-class RISC-V processor implementations,” in Proceedings of the 18th ACM International Conference on ComputingFrontiers, ser. CF ’21. Association for Computing Machinery, 2021, pp. –20. [Online]. Available: https://doi.org/10.1145/3458657

W. Li, T. Liu, Z. Xiao, H. Qi, W. Zhu, and J. Wang, “Tcader: A tightly coupled accelerator design framework for heterogeneous system with hard-ware/software co-design,” Journal of Systems Architecture, vol. 136, p. 102822, 2023. [Online]. Available: https://doi.org/10.1016/j.sysarc.2023.102822

Y. Lee, A. Waterman, R. Avizienis, H. Cook, C. Sun, V. Stojanović, and K. Asanović, “A nm 1.3ghz 16.7 double-precision gflops/w risc-v processor with vector accelerators,” in 40th European Solid State Circuits Conference (ESSCIRC), pp. 199–202. [Online]. Available: https://doi.org/10.1109/ESSCIRC.2014.6942056

A. Amid et al., “Chipyard: Integrated design, simulation, and implementation framework for custom socs,” IEEE Micro, vol. 40, no. 4, pp. 10–21, 2020. [Online]. Available: https://doi.org/10.1109/MM.2020.

C. Danner and M. Kafafy, “Visual chess recognition,” http://web.stanford.edu/class/ee368/Project_Spring_1415/Reports/Danner_Kafafy.pdf, 2015.

J. F. Canny, “Finding edges and lines in images,” Theory of Computing Systems -Mathematical Systems Theory, p. 16, 1983.

R. O. Duda and P. E. Hart, “Use of the hough transformation to detectlines and curves in pictures,” Communications of the ACM, vol. 15, no. 1, p. 11–15. [Online]. Available: https://doi.org/10.1145/361242