In this paper, we derive some sufficient second order optimality conditions for control problems of partial differential equations (PDEs) when the cost functional does not involve the usual quadratic term for the control or higher nonlinearities for it. Though not always, in this situation the optimal control is typically bang-bang. Two different control problems are studied. The second differs from the first in the presence of the L-1 norm of the control. This term leads to optimal controls that are sparse and usually take only three different values (we call them bang-bang-bang controls). Though the proofs are detailed in the case of a semilinear elliptic state equation, the approach can be extended to parabolic control problems. Some hints are provided in the last section to extend the results.