The point-spread-functions (PSFs) of reconstructed images can be deformed due to the detector properties, such as the resolution and depth-of-interaction effects, and data geometries, such as the limited angular coverage in dual-panel PET systems. The PSF deformations cause decreased quantitative accuracy and consistency of the uptake measurements across the field of view. While reconstruction models of the detectors and acquisition process provide improvement in the lesion uptake measurements and uniformity in general, for the limited angle data, or for imperfect reconstruction models, the reconstructed image will still be distorted and in a spatially variant way, thus affecting the quantitative results. We are proposing to use image-based resolution model (IRM) to include such image PSF deformation effects. Originally the IRM was mostly used for approximating data resolution effects in a computationally efficient way, but recently it was also used to mitigate effects of simplified geometric projectors. Our work goes beyond this by including into the IRM reconstruction imperfections caused by the limited angle data and any other (residual) deformation effects. We applied and tested these concepts for a dedicated breast imaging geometry (B-PET) consisting of dual-panel TOF detectors. In this feasibility study, we compared simple spatially invariant approximation to the PSF deformation (by capturing only its general elongation through elongated 3D Gaussian) with the spatially variant model using Gaussian mixture model more accurately capturing asymmetry and shape of the deformed PSF in the simulated B-PET scanner. We tested reconstructions with those models for lesions located at various spatial locations. Results illustrate ability of the IRM to suppress the PSF deformation effects and decrease the overall uptake bias in the reconstruction when using the proposed PSF-based model, and especially making the results more robust independent of the location within the FOV — decreasing the standard deviation of contrast recovery coefficient (CRC) variability between lesions from 12.9% and 14.2% w/o RM and with simple RM, respectively, to 7.0% with the proposed variable IRM.