Spectral audio denoising methods usually make use of the magnitudes of a time-frequency representation of the signal. However, if the time-frequency frame consists of quadrature pairs of atoms (as in the short-time Fourier transform), then the phases of the coefficients also follow a predictable pattern, for which simple models are viable. In this paper, we propose a scheme that takes into account the phase information of the signals for the audio denoising problem. The scheme requires to minimize a cost function composed of a diagonally weighted quadrature data term and a fused-lasso type penalty. We formulate the problem as a saddle point search problem and propose an algorithm that numerically finds the solution. Based on the optimality conditions of the problem, we present a guideline on how to select the parameters of the problem. We discuss the performance and the influence of the parameters through experiments.