In this paper, we propose to exploit the epipolar geometry for dynamic pose-estimation of mobile robots. This is performed using a filtering approach with measurements given by the epipoles. The contribution of the paper is a novel observability analysis using nonlinear and linear tools that leads to achieve an efficient position-based visual servoing (PBVS) approach. Additionally, the visibility constraint problem is solved for this type of approach by using any visual sensor obeying a central projection model. This scheme does not need a target model neither scene reconstruction nor the 3D structure. The effectiveness of the proposed estimation scheme is evaluated via simulations in servoing tasks including obstacles and using omnidirectional vision.