COMPARING THE PERFORMANCE OF CONVOLUTIONAL NEURAL NETWORKS (CNN)، RECURRENT NEURAL NETWORKS (RNN) WITH LSTM، AND FEEDFORWARD NEURAL NETWORKS (FFNN) IN HANDWRITTEN NUMERAL RECOGNITION
Keywords:
Feedforward neural network, Convolutional neural network, recurrent neural network, long short-term memory, handwritten recognitionAbstract
This study aims to compare three types of neural networks in classifying images of handwritten numeral, and the study focuses on FFN (Feedforward Neural Network), CNN (Convolutional Neural Network), RNN (Recurrent Neural Network) with LSTM. The study also aims to compare machine learning and deep learning algorithms. The comparison was carried out using the MNIST dataset, and in addition, an external dataset consisting of 30 images was used to evaluate the performance. Accuracy, Recall, Precision, F1 coefficient, Error rate, Training loss and Validation loss were used to evaluate the results. The results showed that the CNN network achieved the best performance in most of the criteria, followed by the RNN network, and the FFN network showed the lowest performance. The results of this research provide an important guide for selecting the most suitable neural network for handwritten numeral recognition.
References
M. M. Abu Ghosh and A. Y. Maghari, "A Comparative Study on Handwriting Digit Recognition Using Neural Networks," in International Conference on Promising Electronic Technologies, 2017, pp. 1-6, ISBN: 978-1-5386-2269-8/17, DOI: 10.1109/ICPET.2017.20.
H. M. Jabde, C. Patil, S. Mali, and A. Vibhute, "Comparative Study of Machine Learning and Deep Learning Classifiers on Handwritten Numeral Recognition," in International Symposium on Intelligent Informatics, ISI 2022, Smart Innovation, Systems and Technologies, vol. 333, pp. 10, Springer, Singapore, 2023. [Online]. Available: https://doi.org/10.1007/978-981-19-8094-7_10
A. Karim, P. Ghosh, A. A. Anjum, and M. S. Junayed, "A Comparative Study of Different Deep Learning Model for Recognition of Handwriting Digits," in SSRN Electronic Journal, Jan. 2021. DOI: 10.2139/ssrn.3769231.
S. Albahli, F. Alhassan, W. Albattah, and R. U. Khan, "Handwritten Digit Recognition: Hyperparameters-Based Analysis," Applied Sciences, vol. 10, no. 17, p. 5988, 2020.
Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press.
LeCun, Y., Cortes, C., & Burges, C. "MNIST handwritten digit database." Retrieved from http://yann.lecun.com/exdb/mnist/. Accessed on January 27, 2023.
Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning," Nature, vol. 521, no. 7553, pp. 436-444, 2015, DOI:10.1038/nature14539.
Zhang, A., Lipton, Z. C., Li, M., & Smola, A. J. "Dive into deep learning." Retrieved from http://www.d2l.ai. Accessed on October 23, 2022.
C. Zhu, S. Han, H. Mao, and W. J. Dally, "Trained Ternary Quantization," arXiv preprint, arXiv:1612.01064, 2017. [Online]. Available: https://doi.org/10.48550/arXiv.1612.01064.
K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-778, IEEE, 2016.
X. Wang, R. Girshick, A. Gupta, and K. He, "Non-local Neural Networks," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7794-7803, IEEE, 2018.
K. Greff, R. K. Srivastava, J. Koutník, B. R. Steunebrink, and J. Schmidhuber, "LSTM: A Search Space Odyssey," IEEE Transactions on Neural Networks and Learning Systems, vol. 28, no. 10, pp. 2222-2232, 2017.
K. Greff, R. K. Srivastava, J. Koutník, B. R. Steunebrink, and J. Schmidhuber, "LSTM: A Search Space Odyssey," arXiv preprint, arXiv:1503.04069, 2015.
Negnevitsky, M. (2011). Artificial Intelligence: A Guide to Intelligent Systems (3rd ed.). Pearson Education Limited.
Géron, A. (2019). Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow. O'Reilly Media.
Grus, J. (2019). Data Science from Scratch: First Principles with Python (2nd ed.). O'Reilly Media.
N. Ashraf, W. Ahmad, and R. Ashraf, "A Comparative Study of Data Mining Algorithms for High Detection Rate in Intrusion Detection System," Annals of Emerging Technologies in Computing (AETiC), vol. 2, no. 1.
Z. Zhang, Y. Cao, F. Liu, and Y. Zhang, "An Overview of Neural Network Architecture Design for Academic Data Mining," Journal of Educational Technology Development and Exchange (JETDE), vol. 12, no. 1, pp. 1-14, 2019.
G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, "Densely Connected Convolutional Networks," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700-4708, IEEE, 2017.
H. Li, Z. Xu, G. Taylor, and C. Studer, "EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks," in International Conference on Machine Learning, pp. 5256-5266, 2020.
S. Jain and R. Chauhan, "Recognition of Handwritten Digits Using DNN, CNN, and RNN," in Advances in Computing and Data Sciences, pp. 239-248, 2018, doi: 10.1007/978-981-13-1810-8_24.
Downloads
Published
Issue
Section
License
Copyright (c) 2023 Journal of Basic Sciences
This work is licensed under a Creative Commons Attribution 4.0 International License.
