We review the work of our research group over the last 4 years towards the development of efficient parallel finite element algorithms. Target applications are physical problems described by means of non-linear sets of partial differential or integro-differential equations of mixed type, and solved in complex geometries using unstructured finite element meshes. A typical example considered in this paper is the flow of viscoelastic fluids. The complexity of the governing equations is such that it prevents the use of established parallel numerical algorithms developed for elliptic problems. After a brief discussion of viscoelastic governing equations and related sequential numerical techniques, we describe a generic parallel approach to the assembly and solution of finite element equation sets. Automatic load balancing schemes and mesh partitioning methods are discussed. Finally, the proposed algorithms are evaluated in the simulation of viscoelastic Rows described by integral and differential constitutive equations. Results are reported for various distributed memory MIMD parallel computers, including the INTEL IPSC/860 hypercube, the CONVEX Meta Series, and a heterogeneous network of engineering workstations.