The defect detection of road manhole cover is very important for road maintenance and safety. The paper proposes an improved convolutional neural network algorithm to achieve rapid and accurate detection of manhole cover defects. The algorithm improves the activation function model of convolutional neural network. For the Relu activation function, when the input is less than zero, the output is set to zero, which resluts in lossing most of the input information.Therefore,two improved activation functions, MReLu and BReLu, are designed. On this basis, in order to enhance the feature expression ability of neural network model, a two-layer activation function model is proposed. Finally, a large number of comparative experiments were performed on the proposed algorithm in the public data set MNIST, CIFAR-10,and The main parameters of the network are batch size of 32, the maximum number of iterations is 1000, the learning rate is 0.0001, and the attenuation is 50% after 5000 iterations.The experimental results show that the convolutional neural network based on the improved activation function and the application of the two-layer activation function greatly reduces the training parameters, not only the convergence speed is faster, but also can improve the classification accuracy more effectively.