In this study, we propose a method to overcome the main drawback in stochastic computing, low accuracy or related long computing times. Our method exploits dependency in stochastic bit streams with the aid of feedback mechanisms. Accurate (error-free) arithmetic multiplier and adder circuits are implemented. Operations are performed using both stochastic and binary inputs/outputs; binary-stochastic number conversion circuits are implemented for this purpose. We test our circuits by considering performance parameters area, delay, and accuracy. The simulation results are evaluated in a comparison with the results of other stochastic and deterministic (conventional) computing techniques in the literature. Additionally, we discuss the applicability of our method in emerging technologies including printed/flexible electronics for which low transistor counts is desired.