An all-water-based procedure for "controlled" polymer grafting from cellulose nanofibrils is reported. Polymers and copolymers of poly(ethylene glycol) methyl ether methacrylate (POEGMA) and poly(methyl methacrylate) (PMMA) were synthesized by surface-initiated activators regenerated by electron transfer atom transfer radical polymerization (SI-ARGET ATRP) from the cellulose nanofibril (CNF) surface in water. A macroinitiator was electrostatically immobilized to the CNF surface, and its amphiphilic nature enabled polymerizations of both hydrophobic and hydrophilic monomers in water. The electrostatic interactions between the macroinitiator and the CNF surface were studied by quartz crystal microbalance with dissipation energy (QCM-D) and showed the formation of a rigid adsorbed layer, which did not desorb upon washing, corroborating the anticipated electrostatic interactions. Polymerizations were conducted from dispersed modified CNFs as well as from preformed modified CNF aerogels soaked in water. The polymerizations yielded matrix-free composite materials with a CNF content of approximately 1-2 and 3-6 wt % for dispersion-initiated and aerogel-initiated CNFs, respectively.