When dealing with long video data, the task of identifying and indexing all meaningful subintervals that become answers to some queries is infeasible. It is infeasible not only when done by hand but even when done by using latest automatic video indexing techniques. Whether manually or automatically, it is only fragmentary video intervals that we can identify in advance of any database usage. Our goal is to develop a framework for retrieving meaningful intervals from such fragmentarily indexed video data. We propose a set of algebraic operations that includes ourglue join operations, with which we can dynamically synthesize all the intervals that are conceivably relevant to a given query. In most cases, since these operations also produce irrelevant intervals, we also define variousselection operations that are useful in excluding them from the answer set. We also show the algebraic properties possessed by those operations, which establish the basis of an algebraic query optimization.