This paper presents the research on semantics construction for virtual assembly. The reason is that the existing assembly systems are short of understanding of the user's intents and cannot well support 3-dimensional direct manipulations. The method of discretizing and temporalizing the virtual assembly space and behaviors are put forward, and the semantic elements are analyzed. The practices shows that the semantics construction can well support human-computer interaction in virtual assembly and also can sufficiently support assembly process planning.