graph embedding and foreground-background patch pairs to perform object tracking. We first use particle filter to sample some particles. Then we evaluate each particle based on graph embedding and foreground-background patch pairs. For each particle, we use a two-layer model to represent the object, i.e. the inner layer (object layer) and the outer layer (background layer). Both the two layers are divided into patches. We cluster the foreground patches to several classes. Each class forms one sample group pair with the background patches. We perform graph embedding on multiple sample group pairs to discriminate the foreground and the background. Experimental results showed that our method tracked the objects efficiently.