Face anti-spoofing (FAS) is crucial to defense spoofing attack against face recognition system. Most of existing methods use a large number of attack samples to train the classification model, which requires high computational and labelling costs. It’s also not flexible to collect large number of attack sample each time a new attack model is invented. To address the issue, we propose an Attention Auto-Encoder (AAE) based one-class FAS model in this paper. As only real face samples are required for training, the generalization capability of our method can be significantly improved. In addition, for FAS tasks, attention-based model can filter out irrelevant information and pay attention to consistent feature of genuine face. We use reconstruction error and the latent layer of AAE network to calculate the spoofness score to evaluate the proposed approach. Comprehensive experiments on CASIA-FASD and REPLAY-ATTACK databases show that our method achieves superior performance on cross-dataset testing, i.e., 20.0% and 26.9% HTER is achieved. The results suggest that our method is much more robust against attack patterns not available in the training set.