In this letter we show the microstructure of Ta2O5 grown by anodization of TaNx. Transmission electron microscopy images show an amorphous bilayer microstructure in the direction of growth. The bottom layer (that closest to the Si substrate) is characterized by small (<100 Angstrom) nitrogen-rich inclusions embedded in a Ta2O5-matrix. The top layer is amorphous Ta2O5. It was found that the volume fraction of the nitrogen-rich inclusions increased with increasing at.% N and that the average sizes of the inclusions increased Linearly with increasing at.% N of the as-deposited TaN, films. The decrease in dielectric constant of these films is explained by the increase in volume fraction of the nitrogen-rich inclusions. The interface between the TaNx and the Ta2O5 was found to be quite sharp, which may account for the high breakdown voltages of capacitors made from this growth process.