The paper models VSC-HVDC (voltage source converter based HVDC) on rectangular coordinate, on which presents control strategy and designs corresponding regulators. In addition, transformation of symmetric component is used to getting negative sequence components aiming at unsymmetrical instantaneous faults at AC side. On PSCAD/EMTDC, whole control system, including VSC-HVDC, AC system and control parts, is established and runs. The simulating results indicate that controlling goals are achieved and adopted control strategy and regulators take effect. Toward various instantaneous faults concerning symmetric and unsymmetrical, the AC-DC power system responds quickly and possess strong stability.