Insects in vegetable soybean undermine the quality and safety of soybean products. Thus, a non-destructive technique of detecting insect-damaged vegetable soybean must be developed. An efficient detection method based on a hyperspectral image was proposed by selecting the region of interest (ROI) through automatic threshold segmentation and optimal wavelength selection using the fuzzy-rough set model. For the 362 samples of beans, three image features (i.e., entropy, energy, and mean) of the ROI were extracted as classification features, whose spectral region covered 400–1000nm and contained 94 wavelengths. Three or less optimal wavelengths were then selected using a fuzzy-rough set model based on the thermal charge algorithm (FRSTCA). Support vector data description (SVDD) was used to develop classification models for the insect-damaged soybean. For the prediction samples of the beans, the classification results indicated that the normal samples were 100.0% correctly classified using the automatic extracting ROI method based on automatic threshold segmentation. The classification accuracy for the insect-damaged samples was 91.7%, and a 98.8% overall classification accuracy was achieved with the FRSTCA selecting two wavelengths.