Fault tolerant operation can significantly improve the reliability of a Modular Multilevel Converter (MMC), but the precondition is that a fast and accurate fault detection and localization method has been embedded into the MMC control system. This paper proposes a fault diagnosis method used in a distributed control architecture, which makes the local controllers capable of identifying the switching devices in open-circuit fault. Instead of using capacitor voltage, sub-module terminal voltage is measured for capacitor voltage control and fault diagnosis according to different sampling instances. The proposed fault diagnosis based on real-time measurements rather than a complicated algorithm can identify the faulty switches in a few sampling intervals, which is suitable for MMC systems with a large number of sub-modules and distributed control. The multiple switching device faults in different sub-modules can be identified by local controllers independently. Experimental results show that the proposed fault diagnosis method is able to locate faulty switches within 3ms and also immune from false alarms.