This paper develops a model-free H-infinity control design algorithm for unknown linear discrete-time systems by using Q-learning, which is a reinforcement learning method based on an actor-critic structure. In model-free design, there is no known dynamical model of the system. Thus, one has no information on the system matrices, but can access the state variables and input variables. The paper derives an iterative solution algorithm for H-infinity control design that is based on policy iteration. The algorithm is expressed in the form of linear matrix inequalities (LMI) that do not involve the system matrices, but only require data measured from the system state and input. It is shown that, for sufficiently rich enough disturbance, this algorithm converges to the standard H-infinity control solution obtained using the exact system model. Two numerical examples are given to show the effectiveness in obtaining the H-infinity control without any using knowledge of the system dynamics matrices, and the examples show that the results converge to the ones obtained with the exact system dynamics matrices. (C) 2010 Elsevier Ltd. All rights reserved.