We establish local regularity properties for the value function of a variational problem arising in the study of small random perturbations of planar dynamical systems. The approach is to characterize the extremals as solutions to a Hamiltonian system, using the usual Legendre transformation. The differential of the value function is described by a certain stable manifold associated with the Hamiltonian system. The existence and smoothness of this stable manifold is obtained from standard results.