To minimize or upper-bound the value of a function "robustly," we might instead minimize or upper-bound the "epsilon-robust regularization," defined as the map from a point to the maximum value of the function within an epsilon-radius. This regularization may be easy to compute: convex quadratics lead to semidefinite-representable regularizations, for example, and the spectral radius of a matrix leads to pseudospectral computations. For favorable classes of functions, we show that the robust regularization is Lipschitz around any given point, for all small epsilon > 0, even if the original function is non-Lipschitz (like the spectral radius). One such favorable class consists of the semi-algebraic functions. Such functions have graphs that are finite unions of sets defined by finitely many polynomial inequalities, and are commonly encountered in applications.