Enhanced directional fluorescence emission (DFE) is crucial for the optical signal collection by spectrometer/microscopy in biological and medical detection. In this paper, we systematically report the enhanced excitation and DFE of quantum dot (QD) by silver nanoparticles (AgNPs) through theoretical calculations. A finite difference time-domain method is used in the calculations. By changing the size and the number of AgNPs, as well as relative position between QD and AgNPs, we found that when QD is in the middle of two AgNPs (the diameters of two AgNPs are 90 nm and 70 nm, respectively) and the spacing between them is 20 nm (surface to surface), the DFE is enhanced by 70 times compared to the QD on bare glass. The reason for the DFE enhancement is the geometrical scattering and localized surface plasmon resonance (LSPR) created by AgNPs. The enhancement is the largest when the peak wavelengths of geometrical scattering and LSPR are close to each other. Furthermore, we conclude that monolayer AgNPs play a main role in the DFE enhancement of monolayer or low-concentration QDs, and uniform-sized AgNPs are not the best choice for enhancing the DFE. This will be of significance for designing and developing high-sensitivity fluorescent biosensors and high-efficiency QD LEDs.