We report the formation of self-organized porous ZrO2 layers by anodization of Zr in H2SO4 electrolytes. Anodization at 20 V after a potential sweep from open-circuit potential with a defined sweep rate results in tube-like porous ZrO2. In particular, under optimized electrolyte condition and polarization, a highly ordered porous structure is obtained. Furthermore, sponge-like porous ZrO2 is also fabricated under a specific electrochemical condition.