In this paper, a relevant document retrieval method is proposed for document retrieval systems with vector space models (VSM). In recent years, with the size of the database becomes extremely large, there becomes a high demanding of an accurate and fast-time document retrieval algorithm. Based on the maximum similarity criterion, a document retrieval algorithm using the discrete stochastic optimization method is proposed with the user query to retrieve the relevant documents. The proposed algorithm has the self-learning capability for most of the computational effort is spent at the global optimal document and converges fast to the relevant documents in the database. Numerical results demonstrate that the proposed algorithm has a good convergence property and satisfied document retrieval performance in the database.