A new approach based on the generalized Rayleigh quotient for testing the innovation covariance of the Kalman filter is proposed. The optimization process of testing quality is reduced to the classical problem of maximization of the generalized Rayleigh quotient. In the simulations, the longitudinal and lateral dynamics of the F-16 aircraft model are considered, and the detection procedure of sensor/actuator faults, which affect the innovation covariance, is examined. Comparison of the proposed generalized Rayleigh quotients based algorithms for testing the innovation covariance is performed in the sense of the fastest detection of a fault and the detected minimum fault rate. Some recommendations for the fastest detection of the fault are given. (C) 2013 Elsevier Ltd. All rights reserved.