We consider the problem of assigning single-band thermal night vision image with natural day-time color appearance automatically. We present an approach in which supervised learning is first used to estimate colors of monochromic images. Modeling color distribution of thermal imagery is a challenging problem, since there are insufficient local features for estimating the chromatic value at a point. Our model uses a statistical learning algorithm that incorporates multi-scale and spatially arranged image features, and it can be trained on a data set that contains thermal image and registered day-time color image pairs. Experimental results show that our approach leads to relatively accurate description of the desired color distribution and results in thermal images that appear smooth and natural color details, so that the overall scene recognition and situational awareness can be improved.