With the recent interest in physical therapy through sufficient physical activity, considerable efforts have been made to monitor and classify daily human activities, especially for people who need physical rehabilitation. In our previous study, we designed a classifier to identify 25 unique physical activities performed by 92 healthy participants between the ages of 20 and 65. In this study, with the use of a GENEActiv accelerometer to monitor a wide range of daily activities, we present a learning approach to identify unique activities performed by a varied group of participants with various health conditions. The dataset is comprised of 99 senior participants and 23 participants who are significantly taller in height than the general population, performing 8 unique activities. We have extracted 130 different features in time and frequency domain and selected the most efficient features with the sequential forward selection algorithm. With two stages of classification, the first is utilized for combining similar classes, and the second determines the final decision. We have tested two classifiers for our learning approach, the Gaussian mixture models (GMMs) and the hidden Markov models (HMMs) and compared their performances. We have improved the GMM classifier from our previous study and it has shown more promising results for this dataset. We achieved an accuracy of 88.92% when classifying the 8 unique activities with GMM and 93.5% with HMM when classifying 7 activities.