In this paper, we propose a neural network recognizing visual shapes based on the BidirEctional SOM (BESOM) model. The proposed network has 4 features. First, the network is based on the BESOM model, which is a computational model of the cerebral cortex. Second, the Gabor filter, a model of a simple cell in the primary visual area, is used to calculate input features. Third, the network structure mimics the ventral visual pathway of the brain, which is said to recognize visual shapes. Finally, this is the first application of the BESOM model which is large-scale and multi-layer as far as we know. We conducted an experiment to assess the network and confirmed that it can recognize alphabets.