With the ever increasing scale of industrial data, the computational burden for process modeling and analytics has becoming tremendous, particularly for large-scale processes. In this paper, a distributed parallel probabilistic learning framework based on scalable Parameter Server (PS) architecture is proposed for big process data. Under this framework, the traditional Variational Inference (VI) probabilistic model can be transformed to a scalable form through the Stochastic Variational Inference (SVI) algorithm, which can be further deployed on the PS architecture to make a distributed and parallel model. As an example, the traditional Variational Inference Mixture Factor Analysis (VIMFA) model is converted to the SVI-MFA model and deployed on the PS architecture for process big data modeling. Then it is utilized for process monitoring and quality prediction applications. A numerical case is first generated to validate the feasibility and efficiency of the SVI-MFA modeling algorithm, and then a TE benchmark process and a real Methanation Unit process demonstrate the effectiveness of the proposed framework for big process data modeling and analytics. (C) 2019 Elsevier Ltd. All rights reserved.