1. Home
  2. Vol 4, No 1 (2025)
  3. Herath

Evolution and Advancements from Neural Network to Deep Learning

H.M.M.N. Herath(1),


(1) University of Kelaniya
Corresponding Author

Abstract


This systematic literature review explores the recent advancements, applications, and challenges in the field of deep learning. By analyzing a diverse array of primary studies, this review elucidates how deep learning technologies have evolved from simple neural network architectures to complex frameworks capable of transforming various scientific and industrial sectors. Key advancements discussed include significant theoretical developments aimed at enhancing model stability and predictability, the evolution of methodologies to ensure adversarial robustness, and the expansive application of deep learning across different domains such as facial recognition, autonomous navigation, and healthcare. The review also addresses critical challenges faced by the field, including the heavy reliance on large, annotated datasets, the substantial computational demands of advanced models, and the ethical concerns arising from the broader integration of these technologies. The findings suggest that future research should focus on developing more efficient unsupervised and semi-supervised learning techniques, enhancing computational algorithms, and fostering interdisciplinary collaborations to address ethical and practical challenges. This review highlights both the remarkable capabilities and the significant limitations of deep learning, providing insights into its future trajectory in the academic and practical realms.

Keywords


Computational demands; Deep learning, Evolution; Neural network

References


Aggarwal, K., Mijwil, M. M., Al-Mistarehi, A. H., Alomari, S., Gök, M., Alaabdin, A. M. Z., and Abdulrhman, S. H. (2022). Has the future started? The current growth of artificial intelligence, machine learning, and deep learning. Iraqi Journal for Computer Science and Mathematics, 3(1), 115-123.

Aizat, M., Qistina, N., and Rahiman, W. (2023). A comprehensive review of recent advances in automated guided vehicle technologies: Dynamic obstacle avoidance in complex environment toward autonomous capability. IEEE Transactions on Instrumentation and Measurement, 73, 1-25.

Albahar, M. (2023). A survey on deep learning and its impact on agriculture: challenges and opportunities. Agriculture, 13(3), 540.

Alom, M. Z., Taha, T. M., Yakopcic, C., Westberg, S., Sidike, P., Nasrin, M. S., Asari, V. K. (2019). A state-of-the-art survey on deep learning theory and architectures. Electronics, 8(3), 292.

Archana, R., and Jeevaraj, P. E. (2024). Deep learning models for digital image processing: a review. Artificial Intelligence Review, 57(1), 11.

Chen, X., Wang, X., Zhang, K., Fung, K. M., Thai, T. C., Moore, K., and Qiu, Y. (2022). Recent advances and clinical applications of deep learning in medical image analysis. Medical Image Analysis, 79, 102444.

Cheng, L., Gu, Y., Liu, Q., Yang, L., Liu, C., and Wang, Y. (2024). Advancements in accelerating deep neural network inference on AIoT devices: A survey. IEEE Transactions on Sustainable Computing, 9(6), 830-847.

Ching, T., Himmelstein, D. S., Beaulieu-Jones, B. K., Kalinin, A. A., Do, B. T., Way, G. P., and Greene, C. S. (2018). Opportunities and obstacles for deep learning in biology and medicine. Journal of The Royal Society Interface, 15(141), 20170387.

Choudhary, K., DeCost, B., Chen, C., Jain, A., Tavazza, F., Cohn, R., and Wolverton, C. (2022). Recent advances and applications of deep learning methods in materials science. Computational Materials, 8(1), 59.

Dasgupta, S., Wörgötter, F., and Manoonpong, P. (2013). Information dynamics based self-adaptive reservoir for delay temporal memory tasks. Evolving Systems, 4, 235-249.

Di, X., Shi, R., Mo, Z., and Fu, Y. (2023). Physics-informed deep learning for traffic state estimation: A survey and the outlook. Algorithms, 16(6), 305.

Fort, S., Dziugaite, G. K., Paul, M., Kharaghani, S., Roy, D. M., and Ganguli, S. (2020). Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the neural tangent kernel. Advances in Neural Information Processing Systems, 33, 5850-5861.

Fuad, M. T. H., Fime, A. A., Sikder, D., Iftee, M. A. R., Rabbi, J., Al-Rakhami, M. S., and Islam, M. N. (2021). Recent advances in deep learning techniques for face recognition. IEEE Access, 9, 99112-99142.

Javed, H., El-Sappagh, S., and Abuhmed, T. (2025). Robustness in deep learning models for medical diagnostics: security and adversarial challenges towards robust AI applications. Artificial Intelligence Review, 58(1), 1-107.

Kandathil, S. M., Greener, J. G., and Jones, D. T. (2019). Recent developments in deep learning applied to protein structure prediction. Proteins: Structure, Function, and Bioinformatics, 87(12), 1179-1189.

Kitchenham, B., Brereton, O. P., Budgen, D., Turner, M., Bailey, J., and Linkman, S. (2009). Systematic literature reviews in software engineering–a systematic literature review. Information and Software Technology, 51(1), 7-15.

Latendresse, J., Abedu, S., Abdellatif, A., and Shihab, E. (2024). An exploratory study on machine learning model management. ACM Transactions on Software Engineering and Methodology, 34(1), 1-31.

Lee, M. (2023). The geometry of feature space in deep learning models: A holistic perspective and comprehensive review. Mathematics, 11(10), 2375.

Li, Z., Shi, N., Zhao, L., and Zhang, M. (2024). Deep reinforcement learning path planning and task allocation for multi-robot collaboration. Alexandria Engineering Journal, 109, 408-423.

Manakitsa, N., Maraslidis, G. S., Moysis, L., and Fragulis, G. F. (2024). A review of machine learning and deep learning for object detection, semantic segmentation, and human action recognition in machine and robotic vision. Technologies, 12(2), 15.

Martín, A., and Camacho, D. (2022). Recent advances on effective and efficient deep learning-based solutions. Neural Computing and Applications, 34(13), 10205-10210.

Mishra, C., and Gupta, D. L. (2016). Deep machine learning and neural networks: an overview. International Journal Hybrid Information and Technology, 9(11), 401-414.

Najjar, R. (2023). Redefining radiology: A review of artificial intelligence integration in medical imaging. Diagnostics, 13(17), 2760.

Pantoja, M., Behrouzi, A. A., and Fabris, D. (2018). An introduction to deep learning. Concrete International, 40(9), 35-41.

Pati, D., and Lorusso, L. N. (2018). How to write a systematic review of the literature. HERD: Health Environments Research & Design Journal, 11(1), 15-30.

Rane, C., Tyagi, K., Kline, A., Chugh, T., and Manry, M. (2024). Optimizing performance of feedforward and convolutional neural networks through dynamic activation functions. Evolutionary Intelligence, 17(5), 4083-4093.

Rane, N. L., Paramesha, M., Choudhary, S. P., and Rane, J. (2024). Machine learning and deep learning for big data analytics: A review of methods and applications. Partners Universal International Innovation Journal, 2(3), 172-197.

Ribeiro, H. V., Lopes, D. D., Pessa, A. A., Martins, A. F., Da Cunha, B. R., Gonçalves, S., and Perc, M. (2023). Deep learning criminal networks. Chaos, Solitons & Fractals, 172, 113579.

Sapoval, N., Aghazadeh, A., Nute, M. G., Antunes, D. A., Balaji, A., Baraniuk, R., and Treangen, T. J. (2022). Current progress and open challenges for applying deep learning across the biosciences. Nature Communications, 13(1), 1728.

Shujaat, S. (2025). Automated machine learning in dentistry: A narrative review of applications, challenges, and future directions. Diagnostics, 15(3), 273.

Van Dinter, R., Tekinerdogan, B., and Catal, C. (2021). Automation of systematic literature reviews: A systematic literature review. Information and software technology, 136, 106589.

Wang, S., Dong, Z., Zhang, Z., Huo, Y., Celebi, M. E., and Shan, C. (2022). Guest editorial emerging challenges for deep learning. IEEE Journal of Biomedical and Health Informatics, 26(11), 5287-5288.

Wong, A., Bäck, T., Kononova, A. V., and Plaat, A. (2023). Deep multiagent reinforcement learning: Challenges and directions. Artificial Intelligence Review, 56(6), 5023-5056.

Xu, P., Zhu, X., and Clifton, D. A. (2023). Multimodal learning with transformers: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(10), 12113-12132.

Yang, Y. (2023). Current trends in deep learning. Advances in Engineering Technology Research, 5(1), 422-422.

Zhang, T., and Mo, H. (2021). Reinforcement learning for robot research: A comprehensive review and open issues. International Journal of Advanced Robotic Systems, 18(3), 17298814211007305.

Zhou, X., Wang, H., Feng, C., Xu, R., He, Y., Li, L., and Tu, C. (2022). Emerging applications of deep learning in bone tumors: current advances and challenges. Frontiers in oncology, 12, 908873.


Full Text: PDF

Article Metrics

Abstract View : 158 times
PDF Download : 223 times

Refbacks

  • There are currently no refbacks.


Copyright (c) 2025 Bumi Publikasi Nusantara

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.