In this paper, we introduce STAMP - a real-time and low-power sampling error based stochastic number generator - for stochastic computing circuits. STAMP exploits the stochastic nature of sampling error; its name is derived from 'stochastic' and 'sampling'. A unique feature of STAMP which distinguishes it from other random generators is the ability to control the output probability of the generated stochastic bit stream in real-time with no area overhead. STAMP is implemented in 180 nm CMOS. Measurements have shown that STAMP passes all tests in the suit of the National Institute of Standards and Technology (NIST) and outperforms the benchmark random number generators in terms of randomness quality. STAMP performs 140 Mb/s throughput with energy consumption of 77 pJ/bit.