Reconstruction of cross-cut shredded text documents (RCCSTD) plays a crucial role in many fields such as forensic and archeology. To handle and reconstruct the shreds, in addition to some image processing procedures, a well-designed optimization algorithm is required. Existing works adopt some general methods in these two aspects, which may not be very efficient since they ignore the specific structure or characteristics of RCCSTD. In this paper, we develop a splicing-driven memetic algorithm (SD-MA) specifically for tackling the problem. As the name indicates, the algorithm is designed from a splicing-centered perspective, in which the operators and fitness evaluation are developed for the purpose of splicing the shreds. We design novel crossover and mutation operators that utilize the adjacency information in the shreds to breed high-quality offsprings. Then, a local search strategy based on shreds is performed, which further improves the evolution efficiency of the population in complex search space. To extract valid information from shreds and improve the accuracy of splicing costs, we propose a comprehensive objective function that considers both edge and empty row-based splicing errors. Experiments are carried out on 30 RCCSTD scenarios and comparisons are made against previous best-known algorithms. Experimental results show that the proposed SD-MA displays a significantly improved performance in terms of solution accuracy and convergence speed.