We show analytically that in dilute solutions of high molecular weight polymers, a collapse transition of the chain can be induced by proximity to the critical point of the solvent. The transition is driven by the fluctuations in the medium, which lead to an effective attractive interaction of long range between different parts of the polymer. At the critical point itself, however, the chain adopts the same average conformations that characterize its size in the off-critical limit. In other words, on approach to the critical point, the polymer is found first to contract and collapse, and then subsequently to return to its original dimensions. This behavior has recently been observed in simulations of polymer-solvent mixtures near the lower critical solution temperature of the system, and it is also known to be characteristic of solutions of polymers in bicomponent solvent mixtures near the critical consolute point of the two solvents.