Limited natural resources and a continuous increase in the demand for modern technological products, is creating a demand and supply gap for rare earth elements (REEs) and Sc. There is therefore a need to adopt the sustainable approach of the circular economy system (CE). In this review, we defined six steps required to close the loop and recover REEs, using a holistic approach. Recent statistics on REEs and Sc demand and the number of waste generations are reported and studies on more environmentally friendly, economic, and/or efficient recovery processes are summarized. Pilot-scale recovery facilities are described for several types of secondary sources. Finally, we identify obstacles to closing the REE loop in a circular economy and the reasons why secondary sources are not preferred over primary sources. Briefly, recovery from secondary sources should be environmentally and economically friendly and of an acceptable standard concerning final product quality. However, current technologies for recovery from for secondary sources are limiting and technology needs will vary depending on the source type. The quality/purity of the recovered metals should be proven so that they do not result in any adverse effects on the product quality, when they are being used as secondary raw material. In addition, for industrial-scale facilities, process improvements are required that consider environmental conditions.