In this paper, a mathematical model of an oxy-fuel combustion boiler system with flue gas recirculation is investigated. The reduction of CO2 emission from coal-fired power plants is an important research issue in alleviating the global warming. The entire dynamics are decomposed in two main parts; fireside dynamics and water-side dynamics. The fire-side dynamics consist of the mass and energy balance equations in the furnace (combustion process) and the flue gas dynamics represented by the mass balance equations of five gases (O-2, CO2, SO2, H2O and NO2). The water-side dynamics include a drum pressure equation and a steam temperature equation. To validate the developed models, the real experimental data in Karakas. Koumanakos, et al. (2007) are used. To investigate the local behavior near an operating point, a linearization method at its steady-state condition is pursued. The time responses of the entire dynamics using step inputs (the oxygen mass flow rate, the coal mass flow rate, the primary air mass flow rate, etc.) are also discussed. (C) 2010 Elsevier Ltd. All rights reserved.