Neutron stars are thought to be excellent laboratories for determining the equation of state (EoS) of cold dense matter. Their strong gravity suggests that they can also be used to constrain gravity models. The two observables of neutron stars-mass and radius (M-R)-both depend on the choice of EoS and relativistic gravity, meaning that neutron stars cannot be simultaneously good laboratories for both of these questions. A measurement of mass and/or radius would constrain the less well known physics input. The most common assumption-namely, that M-R measurements can be used to constrain the EoS-presumes that general relativity (GR) is the ultimate model of gravity in the classical regime. We calculate the radial profile of compactness and curvature (square root of the full contraction of the Weyl tensor) within a neutron star and determine the domain not probed by the Solar System tests of GR. We find that, except for a tiny sphere of radius less than a millimeter at the center, the curvature is several orders of magnitude above the values present in Solar System tests. The compactness is beyond the solar surface value for r > 10 m, and increases by 5 orders of magnitude towards the surface. With the density being only an order of magnitude higher than that probed by nuclear scattering experiments, our results suggest that the employment of GR as the theory of gravity describing the hydrostatic equilibrium of the neutron stars is a rather remarkable extrapolation from the regime of tested validity, as opposed to that of EoS models. Our ignorance of gravity within neutron stars suggests that a measurement of mass and/or radius constrains gravity rather than the EoS, and given that the EoS has yet to be determined by nucleon scattering experiments, M-R measurements cannot tightly constrain the gravity models either. Near the surface the curvature and compactness attain their largest values, while the EoS in this region is fairly well known. This renders the crust as the best site to look for deviations from GR.