Due to the message transmission between processes in a distributed system, an error in a process might be propagated to another via faulty messages, which causes a global failure. In the absence of built-in fault detection methods, rollback recovery approach is not useful. To avoid error propagation and rollback overhead, roll-forward recovery schemes based on redundancy techniques such as N-Version Programming techniques have been presented. The disadvantage of using these schemes is that they need to block the receiver process until each received message is confirmed by the other version of the process, which results in high time overhead. In the case of variant response latencies, consisting of processing time and message transmission delay, these techniques would not be efficient. In this paper, a non-blocking roll-forward recovery approach with some changes to duplex system is proposed. This approach does not avoid fault propagation. But it performs an additional test using a copy of a failed module version to discover faulty process and replace its state with the fault-free process and mask the faults which are propagated to other processes; so it does not need to block processing or message transmission in any phases of the process. This scheme has lower execution time than existing roll-forward techniques.