Privacy-Preserving Sentiment Analysis Using Homomorphic Encryption and Attention Mechanisms

Homomorphic encryption (HE) is a promising approach to preserving the privacy of data used in machine learning by allowing computations to be performed on ciphertext and exploring ways to achieve faster encrypted neural networks with HE. This paper presents a privacy-preserving sentiment analysis method employing Cheon-Kim-Kim-Song (CKKS) homomorphic encryption [4] on a pre-trained deep learning model. The model is bifurcated into a client-side attention mechanism and a server-side prediction head. The attention mechanism at the client end encrypts pivotal data before transmission, thereby preserving privacy while reducing the computational burden on the server. The server handles this encrypted data with a simplified RNN layer and linear activation function, ensuring computational efficiency without compromising on data privacy. Finally, the client decrypts the server’s encrypted output and applies a sigmoid function to obtain the sentiment score. We demonstrated the efficacy of this approach using the IMDb database [17], achieving an accuracy of 70.73%. This approach maintains a balance between privacy preservation and computational efficiency, showcasing a viable solution for secure and efficient machine learning applications.