Chronic low-grade inflammation (CLGI) is closely associated with various chronic diseases. Researchers have paid attention to the comprehensive application and development of food materials with potential anti-inflammatory activity. Owing to their abundant nutrients and biological activities, coarse cereals have emerged as an important component of human diet. Increasing evidence has revealed their potential protective effects against CLGI in chronic conditions. However, this property has not been systematically discussed and summarized. In the present work, numerous published reports were reviewed to systematically analyze and summarize the protective effects of coarse cereals and their main active ingredients against CLGI. Their current utilization state was investigated. The future prospects, such as the synergistic effects among the active compounds in coarse cereals and the biomarker signatures of CLGI, were also discussed. Coarse cereals show promise as food diet resources for preventing CLGI in diseased individuals. Their active ingredients, including beta-glucan, resistant starch, arabinoxylan, phenolic acids, flavonoids, phytosterols and lignans, function against CLGI through multiple possible intracellular signaling pathways and immunomodulatory effects. Therefore, coarse cereals play a crucial role in the food industry due to their health effects on chronic diseases and are worthy of further development for possible application in modulating chronic inflammation.