q-Neurons: Neuron Activations Based on Stochastic Jackson's Derivative Operators.

Abstract

We propose a new generic type of artificial neurons called q -neurons. A q -neuron is a stochastic neuron with its activation function relying on Jackson's discrete q -derivative for a stochastic parameter q . We show how to generalize neural network architectures with q -neurons and demonstrate the scalability and ease of implementation of q -neurons into legacy deep learning frameworks. We report experimental results that consistently improve performance over state-of-the-art standard activation functions, both on training and test loss functions.