We study the regularity properties of the value function associated with an affine optimal control problem with quadratic cost plus a potential, for a fixed final time and initial point. Without assuming any condition on singular minimizers, we prove that the value function is continuous on an open and dense subset of the interior of the attainable set. As a byproduct we obtain that it is actually smooth on a possibly smaller set, still open and dense.