The authors consider a class of discrete resource allocation problems which are hard due to the combinatorial explosion of the feasible allocation search space. In addition, if no closed-form expressions are available for the cost function of interest, one needs to evaluate or (for stochastic environments) estimate the cost function through direct online observation or through simulation. For the deterministic version of this class of problems, the authors derive necessary and sufficient conditions for a globally optimal solution and present an online algorithm which they show to yield a global optimum. For the stochastic version, they show that an appropriately modified algorithm, analyzed as a Markov process, converges in probability to the global optimum, An important feature of this algorithm is that it is driven by ordinal estimates of a cost function, i.e., simple comparisons of estimates, rather than their cardinal values. They can therefore exploit the fast convergence properties of ordinal comparisons, as well as eliminate the need for "step size" parameters whose selection is always difficult in optimization schemes. An application to a stochastic discrete resource allocation problem is included, illustrating the main features of their approach.