All multimedia devices now incorporate video CODECs that comply with international video coding standards such as H.264 / MPEG4-AVC and the new High Efficiency Video Coding Standard (HEVC) otherwise known as H.265. Although the standard CODECs have been designed to include algorithms with optimal efficiency, large number of coding parameters that can be used to fine tune their operation, within known constraints of for e.g., available computational power, bandwidth, energy consumption, etc. With large number of such parameters involved, determining which parameters will play a significant role in providing optimal quality of service within given constraints is a further challenge that needs to be met. We propose a framework that uses machine learning algorithms to model the performance of a video CODEC based on the significant coding parameters. We define objective functions that can be used to model the video quality, CPU time utilisation and bit-rate. We show that these objective functions can be practically utilised in video Encoder designs, in particular in their performance optimisation within given constraints. A Multi-objective Optimisation framework based on Genetic Algorithms is thus proposed to optimise the performance of a video codec. The framework is designed to jointly minimize the complexity, Bit-rate and to maximize the quality of the compressed video stream.