Memristors have attracted broad interest as a promising candidate for future memory and computing applications. Particularly, it is believed that memristors can effectively implement synaptic functions and enable efficient neuromorphic systems. Most previous studies, however, focus on implementing specific synaptic learning rules by carefully engineering external programming parameters instead of focusing on emulating the internal cause that leads to the apparent learning rules. Here, it is shown that by taking advantage of the different time scales of internal oxygen vacancy (V-O) dynamics in an oxide-based memristor, diverse synaptic functions at different time scales can be implemented naturally. Mathematically, the device can be effectively modeled as a second-order memristor with a simple set of equations including multiple state variables. Not only is this approach more biorealistic and easier to implement, by focusing on the fundamental driving mechanisms it allows the development of complete theoretical and experimental frameworks for biologically inspired computing systems.