Meta-Optical Encoder for Image Segmentation.

Abstract

Deep neural networks achieve remarkable results in image segmentation but are often too computationally intensive for resource-limited devices. We propose a hybrid optical/digital neural network that integrates a meta-optical encoder performing convolution directly in the optical domain, drastically reducing computational complexity. Leveraging knowledge distillation, we compress a traditional U-Net into a compact parallel network and replace its initial convolutional layers with an engineered metasurface. Our approach attains an ∼12000× reduction in parameters and an ∼800× reduction in operations at the cost of an ∼16.6% accuracy loss, while maintaining an ∼81.8% segmentation accuracy. This significantly outperforms fully digital compressed models of similar complexity, which exhibit an ∼31.0% accuracy loss. This work demonstrates the first experimental optical neural network operating on incoherent light capable of complex image segmentation beyond simple classification, paving the way for practical low-power, high-speed hybrid optical/digital systems suitable for autonomous agents and edge devices.