Nonlinear dynamics of radiation-induced oscillatory instability in diffusion flames is numerically investigated by employing a diffusion flame established in stagnant mixing layer with optically thin gas-phase radiation and unity Lewis numbers for all species as a model. Particular attention is focused on the radiation-induced extinction regime that occurs when the Damkohler number is sufficiently large. Transient evolution of the flame, initiated by imposing a Damkohler number perturbation on the steady solution, exhibits three types of flame-evolution behaviors, namely decaying oscillatory solution, diverging solution to extinction, and stable limit-cycle solution. The locus of the critically perturbed Damkohler number, across which diverging solutions are separated from decaying solutions or limit-cycle solutions, is obtained, and it can be used as a dynamic extinction boundary for laminar flamelet library. The bifurcation structure is found to be a double Hopf bifurcation, involving a supercritical Hopf bifurcation and a subcritical Hopf bifurcation. The stable limit-cycle solutions, which occur only in the radiation-induced extinction regime while not observed in the transport-induced extinction regime, are found in a small island-shaped parametric region of Damkohler number and flame temperature, in which the double Hopf bifurcation exists, with perturbation amplitudes smaller than the amplitude of the unstable limit cycle of the subcritical Hopf bifurcation. The stable limit-cycle behavior is implied to be relevant to the remarkably sustainable droplet-flame oscillations observed in the space shuttle experiment.