The unguided transmission nature of wireless communication channels in cognitive radio networks (CRNs) makes it easier for attackers to gather transmitted data or to avoid transmissions when compared to traditional wired systems. Furthermore, the shared spectrum concept may lead to various security threats and misusages, especially in the physical (PHY) layer with attacks such as primary user emulation and spectrum sensing data falsification. In this paper, the common PHY layer attacks are introduced from CRN perspective with the corresponding detection techniques and countermeasures. A correlation based trust factor is defined to identify attacks in CRNs, and a beamforming approach is proposed for their prevention. An optimization problem is defined to determine the beamforming coefficients that minimize the success of especially the spectrum sensing data falsification attack. The performance of the proposed method is compared with non-optimized techniques in different channel conditions and proven to be an efficient countermeasure by simulation results.