Getting network video packets quickly from network traffic is the foundation of network video surveillance and management. For obtaining network video packets immediately from network traffic, this paper presents a fast network video packet classification algorithm. When the network video stream is identified, this algorithm could easily obtain all its following packets from network traffic only by using simple XOR operation, Shift operation and linear Hash function. The experimental results show that the proposed algorithm has better performance than simple rules filter method in higher network traffic.