The stochastic calculus of non-smooth functions indicates that for a continuous semi-martingale X(t), the changes of a function h[ X(t)] at its semi-smooth point (both right-and left-hand side derivatives exist) X(t) = x in [t, t + dt] is at the scale of the local time of X(t), with a mean of the order root dt in the case of Ito processes. We introduce the relative time which evolves at the scale of local time when the semi-martingale is at a semi-smooth point of h(x). The change of h[X(t)] in [t, t + dt] can be precisely measured in the scale of relative time, while this change is wrongly ignored with regular time scale dt. The optimal control problem is well defined with the regular time replaced by the relative time; however, dynamic programming does not seem work well for this problem. We apply the direct-comparison-based optimization approach to the control problem formulated in relative time and derive the generalized Hamilton-Jacobi-Bellman (HJB) equations, which consist of two parts, the classical HJB equation for smooth points, and some additional relations for semi-smooth points. Under some bounded conditions, the optimal value function is the (classical) solution to the generalized HJB equations, and viscosity solution is not needed. In addition, we show that the singular control problem can be formulated and solved in the same framework with the relative time.