Forecasting electricity price allows market participants to make informed and sound decisions. Selecting the best training variables is often involved in forecasting in order to obtain optimal prediction. Support Vector Regression (SVR) provides an effective method to fit data and find minimal risk slack variables around a fit line. The best fit depends on the selected input feature set and the tuning of hyper-parameters. In this paper, various SVM optimization solvers were utilized to train and optimize the SVM parameters to speed up the quadratic programming convergence and increase the forecasting accuracy. Various kernels were used with the SVM Regression model and a comparison of their performance was made. Our experiments demonstrated that the RBF Kernel is the most efficient. The paper contributes to enhancing the quality of SVR electricity price forecasting.