SymGraphAU: Prior knowledge based symbolic graph for action unit recognition

The prior and sample-aware semantic association between facial Action Units (AUs) and expressions, which could yield insightful cues for the recognition of AUs, remains underexplored within the existing body of literature. In this paper, we introduce a novel AU recognition method to explicitly explore both AUs and Expressions, incorporating existing knowledge about their relationships. Specifically, we novelly use the Conjunctive Normal Form (CNF) in propositional logic to express these knowledges. Thanks to the flexible and explainable logic proposition, our method can dynamically build a knowledge base specifically for each sample, which is not limited to fixed prior knowledge pattern. Furthermore, a new regularization mechanism is introduced to learn the predefined rules of logical knowledge based on embedding graph convolutional networks. Extensive experiments show that our approach can outperform current state-of-the-art AU recognition methods on the BP4D and DISFA datasets. Our codes will be made publicly available.