The Peng-Robinson equation of state (PR-EoS) and the Stryjek and Vera modification of the Peng-Robinson equation of state (PRSV-EoS) were used to predict the vapor-liquid equilibrium of the system carbon dioxide(1)/ethanol(2)/water(3) under various temperatures and pressures. We propose a new algorithm that uses a flash calculation to provide a set of initial values and then solves the highly nonlinear phase equilibrium equations using the Broyden modified Newton-Raphson (BNR) method. With interaction parameters for the constituent binary systems, the ternary vapor-liquid equilibrium prediction shows that, at a given temperature, pressure, and water molar fraction in liquid phase, this algorithm is stable to calculate all other compositions. The calculated results from both the PR-EoS and the PRSV-EoS are in good agreement with the experimental data for the binary and ternary systems. A water removal process using supercritical carbon dioxide was experimentally implemented. The pressure dependence of the volume expansion of the mixture ethanol/water with carbon dioxide was also investigated using a high-pressure view cell. The established phase equilibrium calculation was applied to predict the volume expansion behavior and to evaluate the water removal efficiency. Results show that this ternary phase equilibrium calculation is useful to study the water removal process under high pressure carbon dioxide. It also shows that the volume expansion prediction is good at relatively low ethanol mass fraction in the feeding water/ethanol mixture or at low pressures.