Free viewpoint television (FTV) based on ray-space representation was adopted in the MPEG draft because the ray-space can generate an arbitrary viewpoint view without complicated analysis and rendering process. Ray-Space predictive coding is one of main techniques in ray-space based FTV systems, in which block matching and compensation plays a very important role to improve coding performance. However, the computational burden of block matching for ray-space coding is the problem to be solved. Ray-space representation of FTV is introduced and inter-slice correlations of ray-space data are analyzed. Then, a fast adaptive block matching algorithm for ray-space predictive coding is proposed to reduce the encoding complexity. Experimental results show that the proposed block matching method reduce the computational burden and speed up the coding process greatly.