We demonstrate a modulation of thermoelectric power factor via a radial dopant inhomogeneity in B-doped Si nanowires. These nanowires grown via vapor-liquid-solid (VLS) method were naturally composed of a heavily doped outer shell layer and a lightly doped inner core. The thermopower measurements for a single nanowire demonstrated that the power factor values were higher than those of homogeneously B-doped Si nanowires. The field effect measurements revealed the enhancement of hole mobility for these VLS grown B-doped Si nanowires due to the modulation doping effect. This mobility enhancement increases overall electrical conductivity of nanowires without decreasing the Seebeck coefficient value, resulting in the increase of thermoelectric power factor. In addition, we found that tailoring the surface dopant distribution by introducing surface delta-doping can further increase the power factor value. Thus, intentionally tailoring radial dopant inhomogeneity promises a way to modulate the thermoelectric power factor of semiconductor nanowires.