Uncalibrated building energy models, as well as models calibrated only on a single performance indicator such as energy consumption or indoor temperature, can be significantly unreliable regarding model parameters and other performance indicators. The risk of obtaining a calibrated model whose parameters are far from the actual values is particularly high in historic buildings because of the increased uncertainty about the building construction. In this paper, we propose a calibration methodology aimed at reducing this risk and apply it on a medieval building. The building was modeled in EnergyPlus based on an energy audit. A sensitivity analysis was performed to identify significant parameters affecting the errors between simulated and monitored indoor air temperatures. The model was calibrated on the hourly indoor air temperatures in summer by minimizing the root mean square error averaged over the building using a particle swarm optimization algorithm. A second calibration was performed by varying the parameters of a representative room. By comparing the results from these two calibrations, we obtained indications about the accuracy of the model parameters. Finally, the model was validated on hourly indoor air and surface temperatures in winter where temperature root mean square errors ranged from 0.4 to 0.8 K. (C) 2015 The Authors. Published by Elsevier B.V.