A lot of traffic accidents are caused by driver distraction, while most existing driver attention assistance solutions need specialized hardware, which limit their wide accessibility. We present a convenient driver attention detecting system based on smartphones with dual cameras. Firstly, a feature detector describing pupil location, yaw and pitch angle of eyes, position and size of face detected with the front camera is proposed to track drivers gaze direction. Secondly, nine safe gaze areas and one unsafe gaze area are defined. Then a SVM classifier is used to estimate gaze area. At the same time, motion objects in the view of road are detected with the rear camera using optical flow methodology combined dynamic background compensation. At last, an inference module is proposed to quantify the level of drivers visual attention by inferring whether the motion objects are located in the drivers gaze areas. Experimental results on an android pad show that the proposed system can detect driver visual attention in real driving scenarios efficiently.