In this paper, a novel approach is proposed for recursive recovery of non-rigid structures from image sequences captured by a single camera. The main novelty in the proposed method is an adaptive algorithm for construction of shape constraints imposing stability on the online reconstructed shapes. The proposed, adaptively learned constraints have two aspects: constraints imposed on the basis shapes, the basic ¡°building blocks¡± from which shapes are reconstructed; as well as constraints imposed on the mixing coefficients in the form of their probability distribution. Constraints are updated when the current model no longer adequately represents new shapes. This is achieved by means of Incremental Principal Component Analysis (IPCA). The proposed technique is also capable to handle missing data. Results are presented for motion capture based data of articulated face and simple human full-body movement.