Handover (HO) management is essential in any mobile cellular network. It ensures seamless connectivity to the User Equipment (UE) while moving from a Base Station (BS) to another within the coverage area. HO optimization refers to adopting intelligent and automatic HO techniques in mobile networks. HO optimization is taking more importance in the Fifth-Generation (5G) and Beyond (B5G) systems due to the requirements and specifications that B5G targets. The requirements of the B5G, such as global connectivity, ultra-low latency, big data analytics, extreme data rate transmissions, a massive number of devices in a small area, etc., and the new technologies that will support the B5G network, such as Millimeter Wave (mmWave), Terahertz (THz) communication, Ultra-Dense Networks (UDNs), etc. All these cause new HO optimization challenges and require new solutions for HO optimization techniques. This paper comprehensively provides the HO optimization challenges and solutions in B5G. Firstly, it provides a research background and explanation for the HO in legacy. Then, it investigates the HO optimization challenges in B5G, including future research directions. After that, the paper discusses the most prominent and recent techniques and technologies solutions for HO optimization management in B5G. Finally, it highlights the potential techniques for HO optimization in B5G.