In this paper, a speech-interfaced system for fostering group conversations is proposed. The system captures conversation keywords, and shows visual stimuli in a tabletop display. A stimulus can be a feedback to the current conversation or a cue to discuss new topics. This work briefly describes the overall system architecture, and then explains details about the design choices and implementation of the speech processing unit (SPU). The performance of the keyword spotter, part of the SPU, is measured in experiments carried out in a realistic scenario. Results demonstrate that the proposed SPU is suitable for tracking conversation topics and so fostering group conversations by visualizing appropriate stimuli.