Recently, many concrete social infrastructures are beginning to reach the end of their useful lives. Therefore, it is important to evaluate their condition. In this paper, we propose a method of concrete crack direction detection using visual and audio sensors. Firstly, three-dimensional measurement of the concrete surface is performed by using the light-section method. The obtained point cloud data is analyzed and the positions of the cracks are detected. Next, the positions to hit with a hammer are decided based on the crack position information. Finally, the acoustic signal is analyzed to detect the direction of the crack beneath the surface. Experimental results showed that the proposed method accurately detected crack position and crack direction.