Conventional network security control and protective measures based on identity authentication and content authentication can't meet the security requirement of current cloud computing. To address this problem, a behavior authentication mechanism is proposed, which includes establishment of behavior authentication set and behavior authentication strategy. A stochastic Petri net model for this mechanism is constructed. We solve the model by using top-down decomposition and bottom-up hierarchical analysis techniques to decrease the complexity of the model and underling Markov process. This paper further conducts performance evaluation of the system. The research results provide a reference for the design and application of a practical behavior authentication mechanism.