Modern construction projects require sufficient planning and management of resources to become successful. Core issues are tasks that deal with maintaining the schedule, such as procuring materials, guaranteeing the supply chain, controlling the work status, and monitoring safety and quality. Timely feedback of project status aids project management by providing accurate percentages of task completions and appropriately allocating resources (workforce, equipment, material) to coordinate the next work packages. However, current methods for measuring project status or progress, especially on large infrastructure projects, are mostly based on manual assessments. Recent academic research and commercial development has focused on semi- or fully-automated approaches to collect and process images of evolving worksites. Preliminary results are promising and show capturing, analyzing, and documenting construction progress and linking to information models is possible. This article presents first an overview to vision-based sensing technology available for temporary resource tracking at infrastructure construction sites. Second, it provides the status quo of research applications by highlighting exemplary case. Third, a discussion follows on existing advantages and current limitations of vision based sensing and tracking. Open challenges that need to be addressed in future research efforts conclude this paper.