In this paper we present a framework that combines some ideas of the behavioral modeling approach and the prediction error modeling approach. It is shown that the proposed model selection procedure can be rephrased as an optimization problem that only depends on the model parameters. Experiments illustrate the potential of the so-called misfit versus latency framework.