Orthogonal frequency division multiplexing with index modulation (OFDM-IM) employs the indices of the active subcarriers for information transmission, as an alternative to conventional OFDM. It offers high spectral efficiency and high energy efficiency in comparison to OFDM thanks to the information bits conveyed by IM. However, OFDM-IM has the drawback of high peak-to-average power ratio (PAPR) similar to OFDM, and this important problem has not been studied well in the literature. Active constellation extension (ACE), which is one of the well-known PAPR reduction methods, can be used to solve this drawback of OFDM-IM. Owing to the fact that this PAPR reduction method is less effective for OFDM-IM, we propose the extension of the constellation over inactive subcarriers through adding clipped signals over them. These subcarriers have a signal power limited by an upper bound, and this causes a slight degradation in the bit error rate (BER) performance. Computer simulation results demonstrate that our proposed method has a better PAPR reduction performance than the ACE method for OFDM and OFDM-IM while being more energy efficient with a very slight degradation in BER performance when a proper clipping threshold level is selected. Additionally, it is shown that the proposed method and ACE can be further combined, and this provides an improved PAPR reduction. In order to decrease the computational complexity of the PAPR reduction method to the linear-logarithmic level, smart gradient projection (SGP) is employed.