Using high-resolution x-ray scattering, the effect of quenched random disorder (QRD) on the second-order nematic-smectic-A (N-SmA) phase transition in butyloxybenzilidene-octylaniline (4O.8) has been studied. 4O.8 is a nonpolar liquid crystal (LC) with a monomeric smectic-A phase. The QRD is created by aerosil nanoparticles which gelate to form a three-dimensional network, confining the LC. The QRD caused by the aerosil gel generates quenched random fields acting on both the nematic and smectic-A order parameters. This results in the destruction of the quasi-long-range order of the smectic-A phase. The x-ray scattering data are modeled with a structure factor composed of two terms, one thermal and one static, corresponding to the connected and disconnected susceptibilities, respectively. Unlike previous studies, the two parts of the structure factor are decoupled by allowing different thermal and static correlation lengths. Our fitting procedure involves temperature-dependent and temperature-independent (global) variables. The amplitude and the parallel correlation length for the thermal part of the line-shape show critical-like behavior both above and below the transition temperature. Detailed analysis reveals that the thermal correlation length does not truly diverge at the phase transition. This effect is discussed on the basis of a cutoff for the divergence caused by the random fields generated by the aerosil network confining the liquid crystal. The intensity of the static term in the line-shape behaves like the order parameter squared at a conventional second-order phase transition. The effective order parameter critical exponent shows an evolution with increasing aerosil gel density ranging from the Gaussian tricritical value to the 3D-XY value. The results of a pseudocritical scaling analysis are compared to an analysis of 4O.8+aerosil heat capacity data and discussed using a phenomenological correlation between the nematic range of pure liquid crystals and the aerosil mass density, rho(s).