The performance of bulk-heterojunction (BHJ) solar cells is strongly correlated with the nanoscale structure of the active layer. Various processing techniques have been explored to improve the nanoscale morphology of the BHJ layer, e. g., by varying the casting solvent, thermal annealing, solvent annealing, and solvent additives. This paper highlights the role of residual solvent in the "dried" BHJ layer, and the effect of residual solvents on PCBM diffusion and ultimately the stability of the morphology. We show that solvent is retained within the BHJ film despite prolonged heat treatment, leading to extensive phase separation, as demonstrated by the growth in the size and quantity of PCBM agglomerates. The addition of a small volume fraction of nitrobenzene to the casting solution inhibits the diffusion of PCBM in the dry film, resulting in smaller PCBM agglomerates, and improves the fill factor of the BHJ device to 0.61 without further tempering. The addition of nitrobenzene also increases the P3HT crystalline content, while increasing the onset temperature for melting of P3HT side chains and backbone. The melting temperature for PCBM is also higher with the nitrobenzene additive present.