We address the problem of recovering a low-rank matrix that has a small fraction of its entries arbitrarily corrupted. This problem is recently attracting attention as nontrivial extension of the classical PCA (principal component analysis) problem with applications in image processing and model/system identification. It was shown that the problem can be solved via a convex optimization formulation when certain conditions hold. Several algorithms were proposed in the sequel, including interior-point methods, iterative thresholding and accelerated proximal gradients. Based on algorithms from rank minimization and sparse vector recovery, we propose a computationally efficient greedy algorithm that scales better to large problem sizes than existing algorithms.