Hand detection is an important issue in the analysis of drivers activities, assessment of drivers alertness, and subsequent development of driver safety monitoring system. In this work, the hand detection problem is addressed in the deep Convolutional Neural Network (CNN) framework. Hypothesis of hand regions are first generated with high recall rate by AdaBoost detector associated with Aggregated Channel Features (ACF) and then the Convolutional neural networks (CNN) are employed to extract the features of each proposal regions. The CNN was trained via multi-task learning paradigm to detect hand and predict the corresponding bounding box simultaneously. Experiments were conducted on the publically available benchmark VIVA hand detection database, showing marked improvement upon previous works.