中图分类号： TP183
文献标识码： A
DOI： 10.19358/j.issn.2096-5133.2021.02.008
引用格式： 李亚杰，关胜晓，倪长好. 基于X-Linear和语义嵌入的视频描述算法[J].信息技术与网络安全，2021，40(2)：45-51.

Video caption algorithm based on X-Linear and semantic embedding

Abstract： The attention mechanism and video semantic embedding have significantly improved the video description task.In order to make better use of the temporal dynamic features and semantic information of the video,a X-Linear-based semantic embedding video description algorithm(X-Linear Semantic Embedding Network，XLSNet) is proposed. The algorithm is based on a encoder-decoder network and uses the X-Linear attention block to encode video features. This block uses bilinear pooling to increase the high-order interaction of video temporal features, and finally extracts rich temporal dynamic features. In order to make full use of video semantic information, semantically embedded GRU and X-Linear are used as decoders to generate video descriptions. To prevent over-fitting, layer normalization and variational Dropout are used for the GRU of the decoder.The proposed algorithm only uses video frame features, and has achieved good results on the public video description data set MSVD.

Key words : video caption；semantic embedding；X-Linear attention；XLSNet