In this paper emphasis is put on the design of a neural network (NN) to model the direct solar irradiance. Since, unfortunately, a neural network is not a statistician-in-a-box, building a NN for a particular problem is a nontrivial task. As a consequence, we argue that in order to properly model the direct solar irradiance, a systematic methodology must be employed. For this purpose, we propose a two-step approach to building the NN model. The first step deals with a probabilistic interpretation of the NN learning by using Bayesian techniques. The Bayesian approach to modeling offers significant advantages over the classical NN learning process. Among others, one can cite (i) automatic complexity control of the NN using all the available data and (ii) selection of the most important input variables. The second step consists of using a new sensitivity analysis-based pruning method in order to infer the optimal NN structure. We show that the combination of the two approaches makes the practical implementation of the Bayesian techniques more reliable.