In this paper, we present a new framework for blind source separation of temporal correlated signals. In general, temporal correlated signals are not independent which means the independence assumption for independent component analysis method is not satisfied. To achieve good separation performance, we apply high order statistics and temporal structure together to put the separation processing in residual level. The residual signals, which is residual part of source signals by extracted temporal structure, are independent. We discuss two types of BSS problem: instantaneous BSS and convolutive BSS. The cost function is derived by simplifying the mutual information of residual signals for both cases. And then we develop efficient learning algorithms respectively. Computer simulations are given to show the separation performance of the proposed algorithm and some comparisons with other algorithms are also provided.