The efficient condition monitoring and accurate module defect detection in large-scale photovoltaic (PV) farms demand for novel inspection method and analysis tools. This paper presents a deep learning based solution for defect pattern recognition by the use of aerial images obtained from unmanned aerial vehicles. The convolutional neural network is used in the machine learning process to classify various forms of module defects. Such a supervised learning process can extract a range of deep features of operating PV modules. It significantly improves the efficiency and accuracy of asset inspection and health assessment for large-scale PV farms in comparison with the conventional solutions. The proposed algorithmic solution is extensively evaluated from different aspects, and the numerical result clearly demonstrates its effectiveness for efficient defect detection of PV modules.