Buildings worldwide account for a surprisingly high 40% of global energy consumption and produce an increasingly large carbon footprint. Future sustainability of the building sector is therefore strongly dependent on the installation of energy efficient technologies. However, even though these technologies are becoming more and more efficient, human behavior still plays a very important role in the overall building energy consumption. In this paper, a model for occupant behavior within the building in relation to energy consumption, along with a building energy consumption model is proposed based on stochastic Markov models. The energy consumption model is used to predict possible energy saving gains from building retrofitting projects. The objective is to ensure the return on investment from these projects. The obtained results demonstrate that the proposed energy consumption model learns occupant behavioral patterns from the building. Additionally, it reliably reproduces them, predicts the building energy consumption and identifies potential areas of energy waste. (C) 2012 Elsevier B.V. All rights reserved.