This paper presents a task instruction scheme for a service robot operating in a daily life environment. It is difficult for the robot to execute tasks automatically in the daily life environment, because it should handle various objects and the working environment is complex. In such an environment, the concept of shared autonomy by which the robot can share a task with a human is important. The task instruction that we propose is as follows: An operator clicks on a target object in a camera image and the system gets the position of the clicked point in the work space from the corresponding range image. Then, the operator selects the task model and attaches the task information to the position and the robot can perform its task safely and reliably using this task information. The features of the task instruction are as follows. An object model is composed of a list of task models which can be applied to the object category. The user recognizes the object type and its situation in the environment, and instructs a robot to execute a task by selecting a task model according to the object type and situation.
