Computer-aided knee joint vibration signal analysis using the signal processing and machine learning algorithms possesses high potential for the noninvasive detection of articular cartilage degeneration, which helps reduce the frequency of surgical diagnoses. Removal of random noise in knee joint vibration signals is an essential procedure anterior to diagnostic analysis. This paper presents an adaptive filter technique to subtract the random noise from the contaminated knee joint vibration signals. The filter coefficients are adaptively updated by the novel algorithm that minimizes the instantaneous error between the estimated signal power and the desired noise-free signal power. The adaptive filtering results demonstrated that the adaptive filter can effectively eliminate random noise in knee joint vibration signals, leading to a higher signal-to-noise ratio and a faster convergence process than that achieved by the matching pursuit method.