Low complexity probability-based piecewise linear approximation of the sigmoid function

In order to improve the network recognition accuracy in the low complexity condition, a piecewise linear sigmoid function approximation based on the distribution probability of the neurons’ values is proposed only with one addition circuit. The sigmoid function is first divided into three fixed regions. Second, according to the neurons’ values distribution probability, the curve in each region is segmented into sub-regions to reduce the approximation error and improve the recognition accuracy. The slope of the piecewise linear function is set as 2-n, effectively reducing the hardware implementation complexity. Experiments performed on Xilinx’s FPGA-XC7A200T implement the MNIST handwritten digits recognition. The results show that the proposed method achieves a 97.45% recognition accuracy in a deep neural network and 98.42% in a convolutional neural network, up to 0.84% and 0.57% higher than other approximation methods only with one addition circuit.