In this article we use geometric optimal control to completely solve the problem of minimum-time transitions between thermal equilibrium and fixed average energy states of the quantum parametric oscillator, a system which has been extensively used to model quantum heat engines and refrigerators. We subsequently use the obtained results to find the minimum driving time for a quantum refrigerator and the quantum finite-time availability of the parametric oscillator, i.e., the potential work which can be extracted from this system by a very short finite-time process.