Human action recognition plays a vital role in the field of human-robot interaction and is widely researched for its potential applications. In this paper we propose a human action recognition framework for human robot interaction in industrial applications. First, a set of key descriptors are learned from a collection of weak spatio-temporal skeletal joint descriptors using random forests, which reduces the dimensionality and computational effort. We show that our approach reduces the descriptor dimensionality by 61 percent. The key descriptors are used with a multi-label one-versus-all binary random forest classifier for action classification. We propose an extension to the framework that allows recognizing multiple actions for a given time instant. This results in a low latency, flexible and re-configurable method that performs on par with other sophisticated approaches on challenging benchmarks like the MSR Action 3D dataset.