Feature relationships are complex and may contain important information. k top scoring pairs (k-TSP) studies feature relationships by the horizontal comparison. This study examines feature relationships and proposes vertical and horizontal k-TSP (VH-k-TSP) to identify the discriminative feature pairs by evaluating feature pairs based on the vertical and horizontal comparisons. Complexity is introduced to compute the discriminative abilities of feature pairs by means of these two comparisons. VH-k-TSP was compared with support vector machine-recursive feature elimination, relative simplicity-support vector machine, k-TSP and M-k-TSP on nine public genomics datasets. For multi-class problems, one-to-one method was used. The experiments showed that VH-k-TSP outperformed the four methods in most cases. Then, VH-k-TSP was applied to a metabolomics data of liver disease. An accuracy rate of 88.11 ± 3.30% in discrimination between cirrhosis and hepatocellular carcinoma was obtained by VH-k-TSP, better than 77.39 ± 4.10% and 79.28 ± 3.73% obtained by k-TSP and M-k-TSP, respectively. Hence combining the vertical and horizontal comparisons could define more discriminative feature pairs.