Overlay variations occur frequently in lithography process, which should be controlled within the tolerance to guarantee the better pattern resolutions. The operational optimization of overlay aims to predict the unknown overlay variations and compensate them into the wafer production. Due to the uncertain yield, the overlay data for learning are usually incomplete, which makes the overlay optimization very challenging. This paper proposes a novel overlay optimization framework called low-rank manifold optimization (LRMO), which provides new insight to address incomplete overlay data via exploiting low-rank property. First, LRMO can use effectively the correlations from incomplete overlay data, which builds a low-rank model for overlay optimization. In addition, LRMO resorts to Riemannian optimization and designs an efficient algorithm for this low-rank model. The proposed LRMO algorithm analyzes the manifold structure of the overlay data and computes accurate overlay variations with a low computational complexity. The experiments validate that LRMO obtains satisfying performance on the operational optimization of overlay variations. (C) 2017 Published by Elsevier Ltd.