With the development of data processing technology, the number of people using a computer is rapidly growing now. And they came to use it in the everyday life of ordinary homes. However, it is hard for elderly people and disabled persons to treat an interface device such as a mouse and a keyboard. For this reason, the dialog system is needed that permits them to dialog with a computer. A virtual human called avatar is introduced into virtual space. Users can talk to him as if he talked to a real person. When a man talks in real space, the partner chimes in according to a speaker's utterance. By synchronizing utterance with a behavior, a partner can understand a speaker's intention correctly. Moreover, relation between a listener and a speaker is not always fixed in man's conversation, and the relation interchanges. This research proposes how to understand the conversation accompanied with the pointing behavior