This paper aims to discuss a novel variable speed direct drive wind energy conversion system (WECS) based on doubly salient electro-magnetic (DSEM) generator. The basic characteristic of the DSEM generator is firstly introduced. Then, for the relatively low speed of the rotor in large scale direct drive application, some special structures of the DSEM generator are presented. In order to optimize the use of wind energy, the field excitation current is chosen as control variable and the perturb and observe (P&O) algorithm is adopted to achieve the maximum wind energy capture. However, the power disturbance increases with output power in the proposed WECS if the fixed step length P&O algorithm is used. An improved variable step P&O algorithm is proposed to overcome the problem of fixed step length algorithm. Finally, computer simulation is carried out, and the results verify the feasibility of the proposed WECS and its control schemes.