Coexistence curves and turbidity of aqueous solutions of tetraoxyethylene mono-n-octyl ether C8E4 without and with the addition of urea were examined focusing on the critical behavior. The critical exponent beta, which characterizes the shape of coexistence curve, for aqueous C8E4 solution was determined as 0.329 +/- 0.003 in agreement with 3D Ising model. Turbidity measurement for aqueous C8E4 solution also yielded Ising values gamma=1.26 +/- 0.05 and nu=0.63 +/- 0.02 for the critical exponents of the isothermal compressibility and correlation length, respectively. Addition of urea up to 7 M concentration did not modify the critical exponents, although the critical temperature increased by about 20 degrees C with the addition of urea. Coexistence curves of oxyethylene mono-isobutyl ether C4E1 with the addition of 7 M urea also yielded beta=0.326 +/- 0.002 in agreement with 3D Ising model. These are in contrast to our recent results that the critical exponents for heptaoxyethylene mono-n-tetradecyl ether C14E7 in an aqueous salt solution were modified to Fisher's renormalized Ising values by the addition of salt [J. Chem. Phys. 109, 711 (1998)]. The difference in the critical behavior for the nonionic surfactant solutions might be attributed to the solution structure, structure of micellar aggregation of surfactant.