面向轻量级卷积网络的激活函数与压缩模型

因卷积神经网络参数膨胀，导致模型训练时占用大量的计算资源和存储资源，从而限制其在边缘终端上的应用。依据深度可分离卷积模型MobileNet V1的设计思路，结合自门控函数和ReLU函数的特点，构建一种改进的激活函数和压缩神经网络模型MobileNet-rhs。将ReLU函数和swish函数分别作为分段线性函数，设计激活函数ReLU-h-swish，通过优化卷积单元结构，解决模型训练过程中难以激活部分神经元的问题，以减少特征信息丢失。构建一种剔除卷积核的压缩模型，从模型深处自下而上剔除2ⁿ个卷积核，减少逐点卷积的参数量。在CIFAR-10和CIFAR-100数据集上进行实验，结果表明，引入ReLU-h-swish函数构建MobileNet-rhs模型的Top-1分类准确率为80.38%。相比MobileNet-rhs模型，压缩后MobileNet-rhs模型的参数量减少17.9%，其Top-1分类准确率仅降低2.28个百分点。此外，利用Tensorflow将该模型部署在安卓平台上，实现图像分类相册的应用。

Activation Function and Compression Model for Lightweight Convolutional Network

The abundance of computing and storage resources required in model training to relieve the parameter expansion of a deep convolution neural network, limiting a network's application on edge terminals.Based on the design idea of the depthwise separable convolution model MobileNet V1, this study proposes an improved activation function and compressed neural network model that combines the characteristics of the self-gating function and the ReLU function.Taking the ReLU function and swish function as piecewise linear functions, the activation function ReLU-h-swish is designed.By optimizing the convolution unit structure to reduce the loss of feature information, it is difficult to activate some neurons during the process of model training.A compression model with convolution kernels removed is constructed.To compress the model, 2ⁿ convolution kernels are removed from the depths of the model from bottom to top to reduce the number of parameters of point-to-point convolution.The experimental results on the CIFAR-10 and CIFAR-100 datasets show that the Top-1 classification accuracy of the MobileNet-rhs model constructed by introducing the ReLU-h-swish function is 80.38%.Compared to the MobileNet-rhs model, the parameters of the compressed MobileNet-rhs model are reduced by 17.9%, and the Top-1 classification accuracy is reduced by only 2.28 percentage points.In addition, Tensorflow is used to deploy the model on the Android platform, which realizes the application of image classification album.