This paper is concerned with a class of (controlled) singular Volterra integral equations, which could be used to describe problems involving memories. The well-known fractional order ordinary differential equations of the Riemann-Liouville or Caputo types are strictly special cases of the equations studied in this paper. Well-posedness and some regularity results in proper spaces are established for such equations. For an associated optimal control problem, by using a Liapunov type theorem and the spike variation technique, we establish a Pontryagin type maximum principle for optimal controls. Different from the existing literature of optimal controls for fractional differential equations, our method enables us to deal with the problem without assuming regularity conditions on the controls, the convexity condition on the control domain, and some additional unnecessary conditions on the nonlinear terms of the integral equation and the cost functional.