This survey explores the integration of machine learning (ML), deep learning (DL), and reinforcement learning (RL) within wireless communications. It reviews various methods, algorithms, and applications while addressing the challenges and future research directions in this field. The paper highlights the necessity of intelligent techniques to enhance the performance and management of wireless networks, driven by the increasing complexity and demand for higher efficiency. Key areas of focus include network optimization, resource management, security, signal recognition, channel coding, traffic prediction, access control, and energy optimization. The survey also discusses emerging techniques such as federated learning, transfer learning, and multi-agent reinforcement learning, emphasizing their potential to revolutionize wireless communication systems.

1. Bahalul Haque AKM, Zihad OM, Hasan R. 5G and Internet of Things—Integration Trends, Opportunities, and Future Research Avenues. In: 5G and Beyond. Springer; 2023.

2. Rajiv. History of Wireless communication—Morse Code to 5G Technology. Available online: https://www.rfpage.com/history-of-wireless-communication-morse-code-to-5g-technology/ (accessed on 3 December 2024).

3. Rajiv. Evolution of wireless technologies 1G to 5G in mobile communication. Available online: https://www.rfpage.com/evolution-of-wireless-technologies-1g-to-5g-in-mobile-communication/ (accessed on 3 December 2024).

4. Ahmed Solyman AA, Yahya K. Evolution of wireless communication networks: From 1G to 6G and future perspective. International Journal of Electrical and Computer Engineering (IJECE). 2022; 12(4): 3943–3950. doi: 10.11591/ijece.v12i4.pp3943-3950

5. Exploring the Challenges and Limitations of Wireless Communication. Available online: https://www.aal-persona.org/exploring-the-challenges-and-limitations-of-wireless-communication/ (accessed on 3 December 2024).

6. RF Wireless World. Available online: https://www.rfwireless-world.com/Terminology/Advantages-and-Disadvantages-of-wireless-communication.html (accessed on 3 December 2024).

7. Akbar MS, Hussain Z, Ikram M, et al. On Challenges of Sixth Generation (6G) Wireless Networks: A Comprehensive Survey of Requirements, Applications, and Security Issues. Journal of Network and Computer Applications. 2025; 233.

8. Hu S, Chen X, Ni W, et al. Distributed Machine Learning for Wireless Communication Networks: Techniques, Architectures, and Applications. arXiv. 2020.

9. Zhao L, Li S, Tan Z, et al. A Multi-UAV Cooperative Task Scheduling in Dynamic Environments: Throughput Maximization. IEEE Transactions on Computers. 2025; 74(2): 442–454. doi: 10.1109/tc.2024.3483636

10. Zhao L, Li S, Guan Y, et al. Adaptive Multi-UAV Trajectory Planning Leveraging Digital Twin Technology for Urban IIoT Applications. IEEE Transactions on Network Science and Engineering. 2024; 11(6): 5349–5363. doi: 10.1109/tnse.2023.3344428

11. Zhao L, Mao C, Wan S, et al. CAST: Efficient Traffic Scenario Inpainting in Cellular Vehicle-to-Everything Systems. IEEE Transactions on Mobile Computing. 2025; 24(3): 2331–2345. doi: 10.1109/tmc.2024.3492148

12. Mao C, Zhao L, Liu Z, et al. Empowering C-V2X Through Advanced Joint Traffic Prediction in Urban Networks. IEEE Internet of Things Journal. 2024; 11(24): 39780–39793. doi: 10.1109/jiot.2024.3447046

13. Ericsson Mobility Report. Mobile data traffic forecast. Ericsson. 2023.

14. Cisco Annual Internet Report (2018–2023). White Paper. Cisco. 2020.

15. Liu F, Kibalya G, Kumar SVNS, Zhang P. Challenges of Traditional Networks and Development of Programmable Networks. In: Software Defined Internet of Everything. Springer; 2022.

16. Gupta A. Network Management: Current Trends and Future Perspectives. Journal of Network and Systems Management. 2006; 14(4): 483–491. doi: 10.1007/s10922-006-9044-7

17. Li Y. Deep Reinforcement Learning: Opportunities and Challenges. arXiv. 2022.

18. Wong A, Bäck T, Kononova AV, Plaat A. Deep multiagent reinforcement learning: Challenges and directions. Artificial Intelligence Review. 2022; 56(6): 5023–5056. doi: 10.1007/s10462-022-10299-x

19. Umoga UJ, Sodiya EO, Ugwuanyi ED, et al. Exploring the potential of AI-driven optimization in enhancing network performance and efficiency. Magna Scientia Advanced Research and Reviews 2024; 10(1): 368–378.

20. Khoramnejad F, Hossain E. Generative AI for the Optimization of Next-Generation Wireless Networks: Basics, State-of-the Art, and Open Challenges. arXiv. 2024.

21. Piccialli V, Sciandrone M. Nonlinear optimization and support vector machines. Annals of Operations Research. 2022; 314(1): 15–47. doi: 10.1007/s10479-022-04655-x

22. Yang X, Hua Z, Li L, et al. Multi-source information fusion-driven corn yield prediction using the Random Forest from the perspective of Agricultural and Forestry Economic Management. Scientific Reports. 2024; 14(1). doi: 10.1038/s41598-024-54354-9

23. Al-Omari M, Rawashdeh M, Qutaishat F, et al. An Intelligent Tree-Based Intrusion Detection Model for Cyber Security. Journal of Network and Systems Management. 2021; 29(2). doi: 10.1007/s10922-021-09591-y

24. Wang N, Wang P, Alipour-Fanid A, et al. Physical-Layer Security of 5G Wireless Networks for IoT: Challenges and Opportunities. IEEE Internet of Things Journal. 2019; 6(5): 8169–8181. doi: 10.1109/jiot.2019.2927379

25. Wu Y, Khisti A, Xiao C, et al. A Survey of Physical Layer Security Techniques for 5G Wireless Networks and Challenges Ahead. IEEE Journal on Selected Areas in Communications. 2018; 36(4): 679–695. doi: 10.1109/jsac.2018.2825560

26. Melki R, Noura HN, Chehab A, Couturier R. Machine Learning for Physical Layer Security: Limitations, Challenges and Recommendation. 2022 16th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS). 2022; 53–60. doi: 10.1109/sitis57111.2022.00017

27. Zhang H, Yu L, Chen Y, Wei Y. Fast Complex-Valued CNN for Radar Jamming Signal Recognition. Remote Sensing. 2021; 13(15): 2867. doi: 10.3390/rs13152867

28. Donges N, Urwin M. A Guide to Recurrent Neural Networks (RNNs). Built In. 2024.

29. Rossi L, Ajmar A, Paolanti M, Pierdicca R. Vehicle trajectory prediction and generation using LSTM models and GANs. PLOS ONE. 2021; 16(7): e0253868. doi: 10.1371/journal.pone.0253868

30. Seyde T, Werner P, Schwarting W, et al. Solving Continuous Control via Q-learning. arXiv. 2023.

31. Agarwal V, Sharma S. DQN Algorithm for network resource management in vehicular communication network. International Journal of Information Technology. 2023; 15(6): 3371–3379. doi: 10.1007/s41870-023-01399-0

32. Yang Y, Li H, Shen B, et al. Microgrid Energy Management Strategy Base on UCB-A3C Learning. Frontiers in Energy Research. 2022; 10. doi: 10.3389/fenrg.2022.858895

33. Razavi-Far R, Wang B, Taylor ME, Yang Q. In: Federated and Transfer Learning. Springer International Publishing; 2023.

34. Guo W, Zhuang F, Zhang X, Dong J. A comprehensive survey of federated transfer learning: Challenges, methods and applications. Frontiers of Computer Science. 2024; 18(6). doi: 10.1007/s11704-024-40065-x

35. Wei C, Wu G, Barth MJ, et al. KI-GAN: Knowledge-Informed Generative Adversarial Networks for Enhanced Multi-Vehicle Trajectory Forecasting at Signalized Intersections. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 2024; 7115–7124. doi: 10.1109/cvprw63382.2024.00706

36. Pu T, Wang X, Cao Y, et al. Power flow adjustment for smart microgrid based on edge computing and multi-agent deep reinforcement learning. Journal of Cloud Computing. 2021; 10(1). doi: 10.1186/s13677-021-00259-1

37. Rostami SM, Samet S, Kobti Z. A Study of Blockchain-Based Federated Learning. In: Razavi-Far R, Wang B, Taylor ME, Yang Q (editors). Federated and Transfer Learning. Springer; 2023.

38. Luo FL. In: Machine Learning for Future Wireless Communications. Wiley-IEEE Press; 2019.

39. Sun Y, Peng M, Zhou Y, et al. Application of Machine Learning in Wireless Networks: Key Techniques and Open Issues. IEEE Communications Surveys & Tutorials. 2019; 21(4): 3072–3108. doi: 10.1109/comst.2019.2924243

40. Bishop CM. In: Pattern Recognition and Machine Learning. Springer; 2006.

41. Goodfellow I, Bengio Y, Courville A. Deep Learning. MIT Press; 2016.

42. Qin Z, Ye H, Li GY, et al. Deep Learning in Physical Layer Communications. IEEE Wireless Communications. 2019; 26(2): 93–99. doi: 10.1109/mwc.2019.1800601.

43. Ye H, Li GY, Juang BH. Power of Deep Learning for Channel Estimation and Signal Detection in OFDM Systems. IEEE Wireless Communications Letters. 2018; 7(1): 114–117. doi: 10.1109/lwc.2017.2757490.

44. Sutton RS, Barto AG. In: Reinforcement Learning: An Introduction. MIT Press; 2018.

45. Luong NC, Hoang DT, Gong S, et al. Applications of Deep Reinforcement Learning in Communications and Networking: A Survey. IEEE Communications Surveys & Tutorials. 2019; 21(4): 3133–3174. doi: 10.1109/comst.2019.2916583.

46. Di Felice M, Bedogni L, Bononi L. Reinforcement Learning-Based Spectrum Management for Cognitive Radio Networks: A Literature Review and Case Study. Handbook of Cognitive Radio. Springer, Singapore; 2018.

47. Qin Z, Li GY, Ye H. Federated Learning and Wireless Communications. IEEE Wireless Communications. 2021; 28(5): 134–140. doi: 10.1109/mwc.011.2000501

48. Oroojlooy A, Hajinezhad D. A review of cooperative multi-agent deep reinforcement learning. Appl Intell. 2023; 53: 13677–13722. doi: 10.1007/s10489-022-04105-y

49. ERICSSON. Ericsson Intelligent RAN Automation. Available online: https://www.ericsson.com/en/ran/intelligent-ran-automation (accessed on 5 October 2024).

50. Jiang W, Han H, Zhang Y, et al. Graph Neural Networks for Routing Optimization: Challenges and Opportunities. Sustainability. 2024; 16(21): 9239. doi: 10.3390/su16219239.

51. Jiang W. Cellular traffic prediction with machine learning: A survey. Expert Systems with Applications. 2022; 201: 117163. doi: 10.1016/j.eswa.2022.117163.

52. Jiang W. Graph-based Deep Learning for Communication Networks: A Survey. Computer Communications. 2022; 185: 40–54.

53. Available online: https://www.nokia.com/about-us/news/releases/2018/03/27/nokias-new-ai-powered-analytics-software-dramatically-improves customer-experience-and-satisfaction/ (accessed on 5 October 2024).

54. Available online: https://www.ericsson.com/en/cases/2019/augmenting-mimo-energy-management-with-machine-learning-and-ai (accessed on 5 October 2024).

55. Available online: https://www.paloaltonetworks.com/network-security/ngfw-vs-fortinet (accessed on 5 October 2024).

56. ZTE. Industrial 5G Cloud Based Station. Available onlilne: https://www.zte.com.cn/global/solutions_latest/5g-advanced/industrial_cloud_based_station.html (accessed 29 November 2024).

57. HUAWEI. Network Management, Control, and Analysis Software. Available onlilne: https://e.huawei.com/en/products/network-analysis (accessed 29 November 2024).

58. Qualcomm. Available onlilne: https://www.qualcomm.com/news/onq/2022/05/bringing-ai-research-to-wireless-communication-and sensing (accessed 29 November 2024).

59. Pratik K, Amjad RA, Behboodi A, et al. Neural Augmentation of Kalman Filter with Hypernetwork for Channel Tracking. 2021 IEEE Global Communications Conference (GLOBECOM). 2021; 1–6. doi: 10.1109/globecom46510.2021.9685798

60. Kadambi S, Behboodi A, Soriaga JB, et al. Neural RF SLAM for unsupervised positioning and mapping with channel state information. ICC 2022—IEEE International Conference on Communications. 2022; 3238–3244. doi: 10.1109/icc45855.2022.9838367

61. Chen M, Gunduz D, Huang K, et al. Distributed Learning in Wireless Networks: Recent Progress and Future Challenges. IEEE Journal on Selected Areas in Communications. 2021; 39(12): 3579–3605. doi: 10.1109/jsac.2021.3118346

62. Warden P, Situnayake D. TinyML: Machine Learning with TensorFlow Lite on Arduino and Ultra-Low-Power Microcontrollers. O’Reilly Media; 2019.

63. Letaief KB, Shi Y, Lu J, Lu J. Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications. IEEE Journal on Selected Areas in Communications. 2022; 40(1): 5–36. doi: 10.1109/jsac.2021.3126076.