Learned Spherical Image Compression With Spherical Convolution-Self-Attention and Transformer Context Model.

Abstract

Emerging virtual reality (VR) applications bring significant challenges to spherical image compression. Spherical images are first converted into planar images using projections like the equirectangular projection (ERP) to facilitate compression. Methods based on deep neural networks (DNNs) have achieved optimal rate-distortion (R-D) performance in planar image compression. However, the non-uniform sampling of ERP makes the R-D optimization process inefficient when using DNN-based planar compression methods. To address this problem, we propose spherical DNNs for learning based spherical image compression using uniform sampling and ordered rooted tree based index of the Spherical Measure-Based Spherical Image Representation (SMSIR). Specifically, we first define basic spherical operations under the ordered rooted tree based index, including spherical convolution and window transformer, to exploit both local and non-local correlations on the sphere, respectively. We then construct a spherical convolution and a self-attention integrated transformer module named SMixFormer, which simultaneously considers both the enlargement of the receptive fields of local windows and the capture of local and non-local correlations. Furthermore, we introduce a spherical transformer context model with an ordering following the ordered rooted tree based index to enhance the accuracy of the entropy model. To optimize our model, we collect a high-resolution and high-quality spherical image dataset from the Internet. Experimental results demonstrate that our approach outperforms traditional image compression standards, including JPEG, JPEG2000, and BPG. Compared to the learning-based hyperprior planar image compression model, our method achieves a bitrate reduction of over 16%.