Efficient differentially private kernel support vector classifier for multi-class classification

Abstract

In this paper, we propose a multi-class classification method using kernel supports and a dynamical system under differential privacy. For small datasets, kernel methods, such as kernel support vector machines (SVMs), show good generalization performance with high-dimensional feature mapping. However, kernel SVMs have a fundamental weakness in achieving differential privacy because they construct decision functions based on a subset of the training data called support vectors. Furthermore, multi-class SVMs must decompose the training data into a binary class, which requires multiple accesses to the same training data. To address these limitations, we develop a two-phase classification algorithm based on support vector data description (SVDD). We first generate and prove a differentially private SVDD (DP-SVDD) by perturbing the sphere center in a high-dimensional feature space. Next, we partition the input space using a dynamical system and classify the partitioned regions using a noisy count. The proposed method results in robust, fast, and user-friendly multi-class classification, even on small-sized datasets, where differential privacy performs poorly.