This paper addresses the filter design issues for multiuser cooperative nonregenerative MIMO relay systems in both downlink and uplink scenarios. Based on the formulated signal model, the filter matrix optimization is first performed for direct path and relay path respectively, aiming to minimize the mean squared error (MSE). To be more specific, for the relay path, we derive the local optimal filter scheme at the base station and the relay station jointly in the downlink scenario along with a more practical suboptimal scheme, and then a closed-form joint local optimal solution in the uplink scenario is exploited. Furthermore, the optimal filter for the direct path is also presented by using the exiting results of conventional MIMO link. After that, several schemes are proposed for cooperative scenario to combine the signals from both paths. Numerical results show that the proposed schemes can reduce the bit error rate (BER) significantly.