A lattice Boltzmann (LB) method is developed in this article in a combination with X-ray computed tomography to simulate fluid flow at pore scale in order to calculate the anisotropic permeability of porous media. The binary 3D structures of porous materials were acquired by X-ray computed tomography at a resolution of a few microns, and the reconstructed 3D porous structures were then combined with the LB model to calculate their permeability tensor based on the simulated velocity field at pore scale. The flow is driven by pressure gradients imposed in different directions. Two porous media, one gas diffusion porous layer used in fuel cells industry and glass beads, were simulated. For both media, we investigated the relationship between their anisotropic permeability and porosity. The results indicate that the LB model is efficient to simulate pore-scale flow in porous media, and capable of giving a good estimate of the anisotropic permeability for both media. The calculated permeability is in good agreement with the measured date; the relationship between the permeability and porosity for the two media is well described by the Kozeny-Carman equation. For the gas diffusion layer, the simulated results showed that its permeability in one direction could be one order of magnitude higher than those in other two directions. The simulation was based on the single-relaxation time LB model, and we showed that by properly choosing the relaxation time, it could give similar results to those obtained using the multiple-relaxation time (MRT) LB method, but with only one third of the computational costs of MRTLB model.