This paper presents a framework for evaluating quality of video encoding algorithms. This technique works by collecting source videos, encoding the source videos with a given algorithm, and decoding the compressed sequences. The decoded video shall be measured with the original sources in a specific evaluation method and feedback to the algorithm. Under the guidance of this framework, an implementation based on embedded device is achieved. Firstly, we collect test source videos in two ways: the commonly used standard sequences and actual scenes captured by camera. Secondly, a popular video encoding standard, H.264, is realized on TMS320DM368, a digital media system-on-chip produced by TEXAS INSTRUMENTS (TI). Finally, peak signal-to-noise ratio (PSNR) is implemented and computed among different test sequences. In the experiments, we achieve the real time compression of high definition videos with an acceptable PSNR. The video quality framework also can be easily extended to other encoding algorithms.