Abstract: Due to such factors as motion blur, information redundancy, and diverse sign language styles, the current isolated word sign language recognition methods still have limitations in recognition accuracy, background noise resistance, and recognition speed. To address these issues, a novel sign language recognition method based on SlowFast network and enhanced hand attention (EHA-SlowFast) is proposed. This method first employs Yolov5 and DeepSort to detect and track hands, thereby increasing the model’s focus on hand information. Secondly, the Focal loss function is adopted in the backbone network to improve the model’s classification ability. Finally, the SlowFast network structure is improved and a channel spatial attention mechanism is introduced to increase the weight of hand information and suppress background noise interference. Additionally, a keyframe extraction algorithm is proposed, which significantly improves efficiency with some accuracy loss. Experimental results demonstrate that EHA-SlowFast achieves a Top-5 accuracy of 97.79% on the DEVISIGN-D dataset, outperforming other state-of-the-art sign language recognition algorithms.