Stochastic computing (SC) encodes data in the signal probabilities associated with pseudo-random bit-streams. It enables very low-area and low-power arithmetic operations using standard VLSI circuits, it is also highly error-tolerant. While addition, subtraction and multiplication have extremely simple SC implementations, this is not true for division. Known stochastic dividers employ sequential logic circuits whose accuracy, convergence properties, etc., are unsatisfactory or not well under-stood. As a result, division is usually avoided or approximated in SC design. We first review and analyze in depth the existing design approaches to stochastic division. We then propose a novel division technique called CORDIV that exploits correlation between the input parameters. CORDIV not only has lower cost than previous stochastic dividers, but is also significantly more accurate. Area is reduced mainly because CORDIV requires less overhead for stochastic number conversion. We provide experimental data showing a typical 3x reduction in area and about a 10x improvement in accuracy.