We consider a class of controlled queue length processes, in which the control allocates each server's effort among the several classes of customers requiring its service. Served customers are routed through the network according to (prescribed) routing probabilities. In the fluid rescaling, X-n(t) = 1/nX(nt), we consider the optimal control problem of minimizing the integral of an undiscounted positive running cost until the first time that X-n = 0. Our main result uses weak convergence ideas to show that the optimal value functions V-n of the stochastic control problems for X-n(t) converge (as n -> infinity) to the optimal value V of a control problem for the limiting fluid process. This requires certain equicontinuity and boundedness hypotheses on {V-n}. We observe that these are essentially the same hypotheses that would be needed for the Barles-Perthame approach in terms of semicontinuous viscosity solutions. Sufficient conditions for these equicontinuity and boundedness properties are briefly discussed.