The ability to perform accurate localization is a fundamental requirement of the navigation systems intended to guide unmanned ground vehicles in a given environment. Currently, the use of vision-based systems is a very suitable alternative for some indoor applications. This paper presents a novel distributed FPGA-based embedded image processing system for accurate and fast simultaneous estimation of the position and orientation of remotely controlled vehicles in indoor spaces. It is based on a network of distributed image processing nodes, which minimize the amount of data to be transmitted through communication networks and hence allow dynamic response to be improved, providing a simple, flexible, low-cost, and very efficient solution. The proposed system works properly under variable or nonhomogeneous illumination conditions, which simplifies the deployment. Experimental results on a real scenario are presented and discussed. They demonstrate that the system clearly outperforms the existing solutions of similar complexity. Only much more complex and expensive systems achieve similar performance.