Abstract: In order to address the shortcomings of traditional text classification models in incomplete extracting the intrinsic semantic information of short Chinese texts, this paper proposes a text classification model that fuses pre-training models and capsule networks. A multi-scale convolutional neural network is firstly used to extract the local semantics in each layer of different levels of the pre-training model. After that, an attention mechanism is used to fuse the obtained multi-grained local semantics and the global semantics obtained through the capsule network, which is then combined with a regularization method to improve the discrimination ability of the model to the sentiment polarity of the text. Finally, the F1 values of the model in the experiment are compared with the real datasets in three different domains. The experimental results show that the model can extract the semantic features of the short Chinese texts more comprehensively by using the improved capsule network, which improves the accuracy of sentiment polarity discrimination.