Underutilised grain legumes are being promoted as part of crop diversification efforts. However, the lack of comparable information to major legumes is limiting these efforts. The first benchmarking study to compare development and productivity of selected underutilised [bambara groundnut (Vigna subterranea) and cowpea (Vigna inguiculata) and major [groundnut (Arachis hypogaea) and common bean (Phaseolus vulgaris)] grain legumes under varying environments was conducted in KwaZulu-Natal, South Africa during 2015/16 and 2016/17. A completely randomised block design with three replications was used at all sites. Crop phenology, yield, water use (ET) and water productivity (WP) were determined for the crops. Data were analysed separately using ANOVA. Biplot analysis was done using GGE. Bambara groundnut was slow to emerge across sites and seasons (> 211 degree days). Common bean was early maturing (< 1677 degree days) while groundnut and bambara groundnut were late maturing (> 1699 degree days). Yield varied significantly (P < 0.05) across environments and seasons. For all environments, common bean had the lowest ET (208–313 mm); bambara groundnut had the highest ET (437 mm), which was recorded during 2015/16. The highest and lowest WP (0.98 and 0.12 kg m−3, respectively) were observed for groundnut. Cowpea had the most stable WP (0.28–0.38 kg m−3). Based on mean values, the major legumes out-yielded the underutilised grain legumes. Crops behaved differently across different environments. The potential of bambara groundnut was limited to sandy soils. There is need for investments in improving yield of underutilised grain legumes to make them more attractive for crop diversification.