In this paper, we present a scheme to partially encrypt Advanced Video Codec (AVC) encoded video. Content owners and distributors rely on Digital Rights Management (DRM) to protect the content against piracy while delivering the content to the end user. Encrypting the content with a symmetric key is a core technical protection mechanism used by DRM systems. Decrypting video takes up precious computing cycles in devices. This may hamper smooth playback in constrained devices. To address this, we outline a scheme where samples (e.g., video or audio frames) are selectively chosen for encryption. This selection is made at a sample and not at a sub-sample level, i.e., when a sample is chosen for encryption it is encrypted in its entirety. Therefore, the required changes to the decryption process in the player to implement this partial encryption scheme is minimal. In the selection process for encryption, samples are weighted heavily based on the entropy and the sensitivity of the sample. The distribution of high entropy samples vary considerably. Moreover, to cover for cases where the frequency of high entropy samples is low, we introduce checks to ensure a baseline encryption for any sequence within the video. Lastly, we present the results after running this scheme through a set of test AVC content.