Classification of Mouth Action Units: Using Local Binary Patterns

Automated facial expression recognition is still a grand challenge in computer vision. The process of facial expression recognition consists of three main stages after image pre-processing and face acquisition. These stages are extracting the feature vector, which describes the facial image, reducing the dimensionality of that vector if possible, and finally, classifying test images into certain classes after training using the reduced dimensionality vector. A contribution to the first stage is chosen since the key issue in the analysis is finding a descriptive, representative, efficient, and easy to compute feature vector of the facial image. This work lays the ground for building a system that automatically, and in real-time classifies mouth action units (of the facial action coding system) with acceptable rates, and that is illumination tolerant. This is done by proposing the application of local binary patterns (LBPs) to the feature extraction stage of mouth action unit classification. LBPs require no manual pre-processing or initialization, contribute to a real-time system by being computationally simple, are illumination tolerant, and can be used in low-resolution images.