Fast Solver for Interior Point Method of SVM Training by Parallel GMRES and HSS

Abstract

Support Vector Machine (SVM) is one of the latest statistical models for machine learning. The key problem of SVM training is an optimization problem (mainly Quadratic Programming). Interior Point Method (IPM) is one of mainstream methods to solve Quadratic Programming problem. However, when large-scale dataset is used in IPM based SVM training, computational difficulty happens because of computationally expensive matrix operations. Preconditioner, such as Cholesky factorization (CF), incomplete Cholesky factorization and Kronecker factorization, is an effective approach to decrease time complexity of IPM based SVM training. In this paper, we reformulate SVM training into the saddle point problem. As the research question that motivates this paper, based on parallel GMRES and recently developed preconditioner Hermitian/Skew-Hermitian Separation (HSS), we develop a fast solver HSS- pGMRES-IPM for the saddle point problem from SVM training. Computational results show that, the fast solver HSS- pGMRES-IPM significantly increases the solution speed for the saddle point problem from SVM training than the conventional solver CF.