We explore the use of manifold learning as a suitable representation for recognizing visual signs found in Filipino Sign Language. During the learning phase, a reference manifold is derived from a training set of visual signs using Isomap, a non-linear manifold learning algorithm. Individual signs are then projected onto this reference manifold transforming them into trajectories which are compiled into a library. For recognition, the manifold trajectory of an unknown sign is computed and compared with the trajectories found in the library using either Dynamic Time Warping or Longest Common Subsequence Similarity Matching. Experiments using individual uninflected signs achieve recognition rates exceeding 80%.