Combining infinitesimal perturbation analysis (IPA) with stochastic approximation gives identification algorithms to estimate the optimal threshold value for failure-prone manufacturing systems consisting of one machine producing one part type. Two adaptive control schemes are proposed. The adaptive control schemes do not require the knowledge of the distribution functions of the up and down times. Under some appropriate conditions, the strong consistency, as well as the convergence rates, of the identification algorithms and the cost function is established for the adaptive control schemes. In particular, it is shown that central limit theorems hold for the identification algorithms.