A mesoscale diverging combustor with a cylindrical flame holder is developed to improve the flame stability and obtain the steadily symmetrical flame, which can be used as heat source for the mesoscale TPV (Thermophotovoltaic) system. The present work investigates the flammability limits based on equivalence ratio and the flame dynamics in the combustors with/without a cylindrical flame holder. For the combustor without the holder, four typical flame propagation modes, the stable slant, oscillating, spinning x-shape and flame blow-off, are observed, and all these flame patterns are asymmetrical. On the other side, the introduction of the cylindrical flame holder can significantly suppress the unstable flame and improve the flame stability probably due to enhanced flame-wall coupling and/or flow recirculation effects, as expected. The stable flame in the combustor with the holder can remain symmetrical under a quite wide range of equivalence ratio and the spinning x-shape mode does not occur here. Although the oscillating flames are also observed in relative narrow range of equivalence ratio in the combustor with the holder, their behaviors still keep symmetrical and their oscillating frequencies are probably dependent on the enhanced flame-wall coupling effect instead of the flame speed. In addition, the flame blow-off dynamics in the combustors with/without a cylindrical flame holder are also demonstrated and discussed in detail. In summary, the cylindrical flame holder can significantly improve the flame stability and makes the flame always remain symmetrical, which is beneficial for extensive application of the combustor with the diverging structure.