Abstract: Aiming at the problems of many parameters and large calculation of the neural network model for rail defect detection, a lightweight model for rail detection based on knowledge distillation and its training method are proposed. The network model is composed of six layers of convolution layer and three layers of full connection layer. The trained DenseNet model is used as the teacher network to guide the training with the method of knowledge distillation, which makes the training of lightweight model simpler and ensures its accuracy. In the training phase of the model, SAM optimization algorithm with minimum sharpness is added to greatly improve the generalization ability of the model. Then VggNet, ResNet, DenseNet and other models are used as comparative experiments to evaluate the model. The average accuracy of the customized lightweight model trained by knowledge distillation in the rail detection data set is 97.3%, and the model parameter size is only 7.38 M, which is superior to other network models and can be deployed in many mobile terminals.