A phrase recognition method for recognition of street name images is presented in this paper. Some of the challenges posed by the problem are: (i) patron errors, (ii) non-standardized way of abbreviating names, and (iii) variable number of words in a street name image. A neural network has been designed to segment words in a phrase, using distance between components and style of writing. Experiments show perfect word segmentation performance of 85%. Substring matching is attempted only between the main body of a lexicon entry and the word segments of an image. Efforts to reduce computational complexity are successfully made by the sharing of character segmentation results between the segmentation and recognition phases. 83% phrase recognition accuracy was achieved on a test set.