This paper presents a new approach for real-time collaborations in adaptive manufacturing, including web-based remote monitoring and control of an industrial robot, and active collision avoidance for human-robot collaborations. It is enabled by using virtual 3D models driven by real sensor data and depth images of human operators. The objectives of this research are to significantly reduce network traffic needed for real-time monitoring over the Internet and to increase the human safety in a human-robot coexisting environment. The results of a case study show that the approach consumes less than 1% of network bandwidth of traditional camera-based methods, and is feasible and practical as a web-based solution.