We address the numerical approximation of mean field games with local couplings. For power-like Hamiltonians, we consider a stationary system and also a system involving density constraints modeling hard congestion effects. For finite difference discretization of the mean field game system developed in [Y. Achdou and I. Capuzzo-Dolcetta, SIAM J. Numer. Anal., 48 (2010), pp. 1136-1162], we follow a variational approach. We prove that the aforementioned schemes can be obtained as the optimality system of suitably defined optimization problems. In order to prove the existence of solutions of the scheme with a variational argument, monotonicity assumptions on the coupling term are not needed, which allows us to recover general existence results proved by Achdou and Capuzzo-Dolcetta. Next, assuming that the coupling term is nondecreasing, the variational problem is cast as a convex optimization problem, for which we study and compare several proximal-type methods. These algorithms have several interesting features, such as global convergence and stability with respect to the viscosity parameter, which can eventually be zero. We assess the performance of the methods via numerical experiments.