There are many static and dynamic Web services composition strategies, however literatures about automatic composition is very rare. In this article, a new algorithm based on reinforcement learning is proposed to realize web service composition automatically and randomly. On the other hand, the existing composition prototype systems mainly focus on function-oriented composition, but not QoS-oriented composition. After understanding the function-oriented composition by reinforcement learning, this paper then introduces preference logic to seek a QoS optimization solution, which is some kind of qualitative solution. When compared with quantitative solution it has many advantages. The result is a novel algorithm RLPLA, which is an algorithm of Web services composition based on reinforcement learning and preference logic