In this letter, the optical gain mechanism in phototransistor detectors (PTDs) is explored in low light conditions. An analytical formula is derived for the physical limit on the minimum number of detectable photons for the PTD. This formulation shows that the sensitivity of the PTD, regardless of itsmaterial composition, is related to the square root of the normalized total capacitance at the base layer. Since the base total capacitance is directly proportional to the size of the PTD, the formulation shows the scaling effect on the sensitivity of the PTD. Our proposedmodel can be used to explore a wide range of PTDs, including nanowire, monolayer, and bulk devices. For the illustration of the ability of the model, two different PDTs, one with bottom–up fabrication for ultraviolet and the other with top–down fabrication for short-wave infrared (SWIR) detection, are used. For the SWIR PTD, a scaling effect is explored on InGaAs-based devices. Our modeling predicts that this PTD with a nanoscale electronic area can reach to a single photon noise equivalent power even at room temperature. To the best of our knowledge, this is the first comprehensive study on the sensitivity of the PTD for extreme low light detection.