A statistical mechanical formalism based on stochastic complexity is presented. Stochastic complexity is a statistical estimation scheme that uses the principle of minimal description length (MDL). The basic concept is that the best form of statistical estimation is one in which both the data and the model＇s structure and parameters are represented in the shortest binary string. This scheme is a generalization that encompasses both the maximum likelihood and maximum entropy methods. To apply this scheme to thermodynamic systems, the minimal number of bits required to describe both the observables and the phase space coordinates of the system is determined. The entropy of a microcanonical ensemble is associated with this minimum description length. This formulation provides a means for calculating partition functions for nonequilibrium systems that are strongly nonergodic. Because of the Bayesian properties of this estimation scheme, techniques for "image reconstruction＇＇ of phase space can be developed. These allow partition functions to be calculated from computer simulation data.