The solution of most nonlinear control problems hinges upon the solvability of partial differential equations or inequalities. In particular, disturbance attenuation and optimal control problems for nonlinear systems are generally solved exploiting the solution of the so-called Hamilton-Jacobi (HJ) inequality and the Hamilton-Jacobi-Bellman (HJB) equation, respectively. An explicit closed-form solution of this inequality, or equation, may however be hard or impossible to find in practical situations. Herein we introduce a methodology to circumvent this issue for input-affine nonlinear systems proposing a dynamic, i.e., time-varying, approximate solution of the HJ inequality and of the HJB equation the construction of which does not require solving any partial differential equation or inequality. This is achieved considering the immersion of the underlying nonlinear system into an augmented system defined on an extended state-space in which a (locally) positive definite storage function, or value function, can be explicitly constructed. The result is a methodology to design a dynamic controller to achieve L-2-disturbance attenuation or approximate optimality, with asymptotic stability.