This paper investigates adaptive optimal control of a grid-independent photovoltaic system consisting of a collector storage, and a load. The control algorithm is based on Q-Learning, a model-free reinforcement learning algorithm, which optimizes control performance through exploration. Q-Learning is used in a simulation study to find a policy which performs better than a conventional control strategy with respect to a cost function which places more weight on meeting a critical base load than on those non-critical loads exceeding the base load.