Motivated by the study of the directed polymer model with mobile Poissonian traps or catalysts and the stochastic parabolic Anderson model with time-dependent potential, we investigate the asymptotic behavior of $$\begin{aligned} \mathbb {E}\otimes \mathbb {E}_0\exp \left\{ \pm \ \theta \int \limits ^t_0\bar{V}(s,B_s)\hbox {d}s\right\} \quad (t\rightarrow \infty ) \end{aligned}$$ E ⊗ E 0 exp ± θ ∫ 0 t V ¯ ( s , B s ) d s ( t → ∞ ) where $$\theta >0$$ θ > 0 is a constant, $$\overline{V}$$ V ¯ is the renormalized Poisson potential of the form $$\begin{aligned} \overline{V}(s,x)=\int \limits _{\mathbb {R}^d}\frac{1}{|y-x|^p}\left( \omega _s(\hbox {d}y)-\hbox {d}y\right) , \end{aligned}$$ V ¯ ( s , x ) = ∫ R d 1 | y - x | p ω s ( d y ) - d y , and $$\omega _s$$ ω s is the measure-valued process consisting of independent Brownian particles whose initial positions form a Poisson random measure on $$\mathbb {R}^d$$ R d with Lebesgue measure as its intensity. Different scaling limits are obtained according to the parameter $$p$$ p and dimension $$d$$ d . For the logarithm of the negative exponential moment, the range of $$\frac{d}{2}<p<d$$ d 2 < p < d is divided into five regions with various scaling rates of the orders $$t^{d/p}$$ t d / p , $$t^{3/2}$$ t 3 / 2 , $$t^{(4-d-2p)/2}$$ t ( 4 - d - 2 p ) / 2 , $$t\log t$$ t log t and $$t$$ t , respectively. For the positive exponential moment, the limiting behavior is studied according to the parameters $$p$$ p and $$d$$ d in three regions. In the subcritical region ( $$p<2$$ p < 2 ), the double logarithm of the exponential moment has a rate of $$t$$ t . In the critical region ( $$p=2$$ p = 2 ), it has different behavior over two parts decided according to the comparison of $$\theta $$ θ with the best constant in the Hardy inequality. In the supercritical region $$(p>2)$$ ( p > 2 ) , the exponential moments become infinite for all $$t>0$$ t > 0 .