In this paper, the consensus problem of hierarchical multi-agent systems has been addressed. A novel approach is employed to divide the agents into different groups with different number of agents. In each group, agents communicate with each other under a constant delay. However, the groups in the upper layer interact with each other with a time-varying delay. The topologies of the groups are also considered to be different. In order to prove the stability of the system, a Lyapunov–Krasovskii functional is employed, and proper conditions for the consensus of this hierarchical system are provided via LMIs. Finally, the results of this research are validated through simulations.