Joint regularized nearest points for image set based face recognition

Face recognition based on image set has attracted much attention due to its promising performance to overcome various variations. Recently, classifiers of regularized nearest points, including sparse approximated nearest points (SANP), regularized nearest points (RNP) and collaborative regularized nearest points (CRNP), have achieved state-of-the-art performance for image set based face recognition. From a query set and a single-class gallery set, SANP and RNP both generate a pair of nearest points, between which the distance is regarded as the between-set distance. However, the computing of nearest points for each single-class gallery set in SANP and RNP ignores collaboration and competition with other classes, which may cause a wrong-class gallery set to have a small between-set distance. CRNP used collaborative representation to overcome this shortcoming but it doesn't explicitly minimize the between-set distance. In order to solve these issues and fully exploit the advantages of nearest points based approaches, in this paper a novel joint regularized nearest points (JRNP) is proposed for face recognition based on image sets. In JRNP, the nearest point in the query set is generated by considering the entire gallery set of all classes; at the same time, JRNP explicitly minimizes the between-set distance of the query set and a single-class gallery set. Furthermore, we proposed algorithms of greedy JRNP and adaptive JRNP to solve the presented model, and the classification is then based on the joint distance between the regularized nearest points in image sets. Extensive experiments were conducted on benchmark databases (e.g., Honda/UCSD, CMU Mobo, You Tube Celebrities databases, and the large-scale You Tube Face datasets). The experimental results clearly show that our JRNP leads the performance in face recognition based on image sets.