In this paper, the lateral response of unreinforced masonry walls (URM) subjected to different levels of axial load is studied numerically and experimentally. The walls in general are noted to exhibit increasing levels of lateral strength when the magnitude of applied compressive axial force is increased. Beyond a certain threshold magnitude of axial compression, this trend in the URM interaction diagram is then noted to be reversed in which the wall exhibits a weaker lateral response as the axial force exerted on the wall increases. An experimental program was conducted to study the lateral load response of sandstone and limestone walls. The finite element simulation using a plastic-damage model in terms of parameters that account for hardening and softening of the wall components in an ABAQUS environment was conducted, and the relationship between lateral and axial force that encompasses the entire spectrum of failure modes was established. The model was tested against available experimental data, and the interaction between lateral and axial force has been expressed in a ready to use form. The results obtained from the plastic-damage-based FEM simulation have also been compared to existing mechanistic models.