Model-based design of separation processes for heterogeneous azeotropic mixtures is a challenging task. The multiplicity of the solutions of equilibrium calculations and the discontinuity due to the potential switching between homogeneous and heterogeneous mixtures on a tray severely complicate the application of deterministic optimization. This paper presents a novel computational approach to reliably determine phase stability and the correct equilibrium solutions in every iteration of the optimization. The approach builds on the decomposition of the optimization problem into a generic superstructure model and an implicit model for equilibrium and enthalpy calculations, which is integrated into the optimization problem by means of an external function. The phase states and equilibrium solutions are determined by means of a reliable homotopy continuation algorithm. An additional reformulation of equilibrium solutions and enthalpy calculations allows overcoming the discontinuity problems. Different case studies illustrate the applicability and show the potential of the proposed method. (C) 2014 Elsevier Ltd. All rights reserved.