Cabled observatory video data are a rich source of information for marine biologists. However, the large amount of recorded video creates a “big data” problem, which calls for automated detection techniques for sea life. Most of the related research has addressed detection based on animal motion. We propose a novel method for the detection of stationary animals, more precisely for crabs. Our approach integrates shape and color information for the automatic detection of crab and non-crab image patches. With these patches we train a feed-forward neural network, which is further used for classifying image patches into crab and non-crab classes. The experimental evaluation shows very promising results.