With their autonomous operation and low environmental impact, solar photovoltaics (PV) are an attractive power source for off-grid systems. However, the variable nature of solar energy is not well-suited to power conventional loads. Without careful consideration of the time-dependent power generation of PV, this discrepancy results in systems that are either over-designed and expensive, or compromise reliability. To accelerate the adoption of PV into new areas, it is essential to design PV-powered systems that are persistent, predictable, and affordable. In this paper, we analyze the cost reductions enabled by design optimization through time flexible operation and improved load sizing. We consider two cases: (i) an idealized reference system, operating 8 h per day at 1 kW, generating an unspecified accumulable output, and (ii) a village-scale PV-powered electrodialysis desalination system, designed to generate 10 m(3) of drinking water per day. We found that time flexible load operation reduced the power system cost of the idealized reference system by 39%, from $2662 to $1628, and designing its electrical load to operate for an optimal period of time enabled an additional cost reduction of 5% (to $1503). For the village-based desalination system, we found that flexible operation paired with expected large decreases in membrane cost (from $150 to $20 per unit) reduced the associated power system cost by 57.6% from $8935 to $3788.