In this paper, an innovative method of advanced data preprocessing for rule extraction using mixture principal component analysis (PCA) models is proposed to build the fault diagnosis system. The experimental data are properly transformed and classified by heuristic smoothing clustering (HSC) and PCA. HSC is a very good method for smoothing measurement observations and characterizing the local Variations of the observations. This heuristic building rule is able to derive fuzzy rules quickly from a set of local regions. Then, the expectation-maximization algorithm is used to tune the corresponding rule parameters for estimating maximum likelihood parameters of PCA models without solving the highly nonlinear coupled equations. The fuzzy rules are constructed by hyperellipsoids of the PCA model whose principal axes are parallel to the major directions of system data distribution; This research is to create an alternative knowledge acquisition methodology based on the mathematical type to describe the elicited knowledge. Results obtained using the new method present the capability of determining the set of conditions for the artificially created data and the well-known reactor system.