Kinect application is becoming a research focus in the field of computer vision. The latest 2.0 version is better than that of 1.0 at the geometric quality and the data transmission. For position estimation and 3D modelling, the popular methods widely employ single data source (depth images or color images) and rarely integrate them, thus providing less robust and precise registration. This paper proposes a new approach for the registration of depth data with color data, which combines epipolar constraints with point-to-plane constraints to improve ICP algorithm and achieve accurate registration. Based on theoretical analysis and experimental verification, results demonstrate the potential of this method, even in a scene tending to be flat where KinectFusion fails in tracking and modelling. The registration accuracy of point cloud is also found to be accord with the observation accuracy of kinect.