This paper is concerned with some long-run average cost problems for controlled Markov chains with a denumerable state space. The criterion to be optimized contains both reward and penalty functions. As a trade-off between reward and penalty, we observe certain phase transition phenomena. Our results also provide a stochastic optimal control interpretation for phase transitions of discrete homopolymers with finite attracting potentials.