Due to the semantic gap between the insufficient facial features and facial identifying information, the single sample per person (SSPP) problem has always been a significant challenge in the field of facial recognition. To address this problem, this paper proposes a Self-Organizing Map (SOM)-based binary coding (SOM-BC) method, which extracts the middle-level semantic features by merging the SOM network with the Bag-of-Features (BoF) model. First, we extract the local features of the facial images using the SIFT descriptor. Next, inspired by human visual perception, we utilize a SOM neural network to obtain a visual words dictionary capable of reflecting the intrinsic structure of facial features in semantic space. Subsequently, a binary coding method is further proposed to map the local features into semantic space. Finally, we propose a simple but effective similarity measure method for classification. Experimental results on three public databases not only demonstrate the effectiveness of the proposed method, but also its high computational efficiency.