Self-assembled InxGa1 As-x quantum dots (QDs) are promising hosts for spin qubits with excellent coupling to photons. Nuclear spin and charge fluctuations lead to dephasing and limit the applicability of QDs as qubits. We show that charge noise can be minimized by high quality MBE growth of well-designed heterostructures yielding natural optical linewidths down to 1.15 mu eV. To minimize the nuclear spin noise, one direction would be to reduce the wave function overlap with the nuclei. We show that this is indeed the case for a single hole spin in a QD that we embedded in the intrinsic region of an n-i-p-diode. For random nuclei, the heavy-hole limit is achieved down to neV energies, equivalent to dephasing times of microseconds. (C) 2017 Elsevier B.V. All rights reserved.