Air conditioning in recent BEMS is now being highly automated, but providing true thermal comfort to building occupants and visitors is still challenging due to difficulty of sensing whole 3D space with limited number of sensors. In particular, thermal effect by crowd of people has not been considered so far in such automated air conditioning. This paper presents a method to accurately estimate the thermal comfort of building occupants considering the thermal effect by human crowd. We modify the PMV index, an well-known thermal comfort index, to reflect such effect. This model is further modified to reduce the model parameters, focusing on the behavior and environmental characteristics. Computational fluid dynamics (CFD) simulations have been employed to build the new PMV model. The model was examined in a real field experiment.