Evolutionary algorithms and nature-inspired optimization algorithms are widely used in solving nonlinear optimization problems. Considering a D-dimensional space, this paper introduces a new random search algorithm called vector-based swarm optimization (VBSO). In this method, vectors with appropriate orientation gradually converge to a global optimum point. In the VBSO, random weighting coefficients are used with a predetermined strategy. Using multiplication of these coefficients by suitable vectors, the randomness property is provided. Taking same conditions into account, the proposed algorithm is compared with a number of well-known intuitive algorithms. Considering 29 unimodal and multimodal benchmark functions, simulation results confirm that the VBSO performs faster and more accurate than intuitive algorithms, such as CEP, FEP, GA, PSO, GSA, DE and ODE, in most cases. To evaluate the performance of the proposed algorithm in coping with difficult situations, some challenging cases are considered. They may have large number of variables, few number of populations, few number of iterations, or may be shifted functions with huge number of optimums. Overall, the simulation results reveal that the performance of the VBSO is satisfactory for such challenging cases.