STSG: A Short Text Semantic Graph Model for Similarity Computing Based on Dependency Parsing and Pre-trained Language Models

References

• Achiam, J., S. Adler, S. Agarwal, L. Ahmad, I. Akkaya. 2023 Dec 19. Gpt-4 technical report. arXiv Preprint arXiv 2303:08774.
   Google Scholar
• Alex, W., S. Amanpreet, M. Julian, H. Felix, L. Omer, and B. Samuel. 2019 Feb 22. GLUE: A multi-task benchmark and analysis platform for natural language understanding. arXiv Preprint arXiv 1804:07461.
   Google Scholar
• Ashish, V., S. Noam, P. Niki, U. Jakob, J. Llion. 2017. Attention is all you need. Proceedings of the 31st Annual Conference on Neural Information Processing Systems, Long Beach.
   Google Scholar
• Binyuan, H., G. Ruiying, W. Lihan, Q. Bowen, L. Bowen. 2022. S2SQL: Injecting syntax to question-schema interaction graph encoder for text-to-SQL parsers. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, 1254–29. Dublin.
   Google Scholar
• Chandrasekaran, D., and V. Mago. 2021. Evolution of semantic similarity—A survey. ACM Computing Surveys 54 (2):1–37. doi:10.1145/3440755.
   Web of Science ®Google Scholar
• Dazhi, J., W. Runguo, H. Zhihui, L. Senlin, L. Cheng, and Y. Lin. 2023. GASN: Gamma distribution test for driver genes identification based on similarity networks. Connection Science 35 (1):1–19. doi:10.1080/09540091.2023.2167937.
   Web of Science ®Google Scholar
• Devlin, J., M. W. Chang, K. Lee, and K. Toutanova. 2018 Oct 11. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv Preprint arXiv 1810:04805.
   Google Scholar
• Ghafour, A., M. Jamshid Bagherzadeh, and F. Mohammad-Reza. 2022. Learning bilingual word embedding mappings with similar words in related languages using GAN. Applied Artificial Intelligence 36 (1). doi:10.1080/08839514.2021.2019885.
   Web of Science ®Google Scholar
• Gizem, S., Ö. Hakime, and Ö. Arzucan. 2017. BIOSSES: A semantic sentence similarity estimation system for the biomedical domain. Bioinformatics 33 (14):I49–58. doi:10.1093/bioinformatics/btx238.
   Web of Science ®Google Scholar
• Guimin, C., T. Yuanhe, S. Yan, and W. Xiang. 2021. Relation extraction with type-aware map memories of word dependencies. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, 2501–12. Bangkok.
   Google Scholar
• He, P., X. Liu, J. Gao, and W. Chen. 2021 Oct 6. DeBERTa: Decoding-enhanced BERT with disentangled attention. arXiv Preprint arXiv 2006:03654.
   Google Scholar
• Hironori, T., S. Junya, F. Sulfayanti, and K. Akihiro. 2022. Anomaly detection using siamese network with attention mechanism for few-shot learning. Applied Artificial Intelligence 36 (1). doi:10.1080/08839514.2022.2094885.
   Web of Science ®Google Scholar
• Ilya, L., and H. Frank. 2019 Jan 4. Decoupled weight decay regularization. arXiv Preprint axXiv 1711:05101.
   Google Scholar
• Jianguo, C., L. Kenli, B. Kashif, Z. Xu, L. Keqin. 2019. A Bi-layered parallel training architecture for large-scale convolutional neural networks. IEEE Transactions on Parallel and Distributed Systems 30(5):965–76. doi:10.1109/TPDS.2018.2877359.
   Web of Science ®Google Scholar
• Jonas, M., and T. Aditya. 2016. Siamese recurrent architectures for learning sentence similarity. Proceedings of the AAAI conference on artificial intelligence, 2786–92. Arizona.
   Google Scholar
• Joshi, M., D. Chen, Y. Liu, S. Weld Daniel, L. Zettlemoyer, Zettlemoyer, L. and Levy, O. 2020. SpanBERT: Improving pre-training by representing and predicting spans. Transactions of the Association for Computational Linguistics 8:64–77. doi:10.1162/tacl_a_00300.
   Google Scholar
• Kevin, C., L. Minh-Thang, V. Quoc, and M. Christopher. 2020 May 23. ELECTRA: Pre-training text encoders as discriminators rather than generators. arXiv Preprint arXiv 2003:10555.
   Google Scholar
• Lei Jimmy, B., K. Ryan, E. Geoffrey, H. B. Jimmy Lei. 2016 Jul 21. Layer normalization. arXiv Preprint arXiv 1607:06450.
   Google Scholar
• Liang, Y., M. Chengsheng, and L. Yuan. 2019. Graph convolutional networks for text classification. Proceedings of the AAAI conference on artificial intelligence, 7370–77. Hawaii.
   Google Scholar
• Lin, D. 1998. An information-theoretic definition of similarity. Proceedings of the International Conference on Machine Learning, 296–304. Madison.
   Google Scholar
• Lingling, X., X. Haoran, W. Fu Lee, T. Xiaohui, W. Weiming, and Q. Li. 2024. Contrastive sentence representation learning with adaptive false negative cancellation. Information Fusion 102:102065–102065. doi:10.1016/j.inffus.2023.102065.
   Web of Science ®Google Scholar
• Mary, P., and H. Marcus. 2022 Jul 19. Formal algorithms for transformers. arXiv Preprint arXiv 2207:09238.
   Google Scholar
• Mengting, H., Z. Xuan, Y. Xin, J. Jiahao, Y. Wei, and C. Gao. 2021. A survey on the techniques, applications, and performance of short text semantic similarity. Concurrency and Computation: Practice and Experience 33 (5). doi:10.1002/cpe.5971.
   Web of Science ®Google Scholar
• Minh Hieu, P., and O. Philip. 2020. Modelling context and syntactical features for aspect-based sentiment analysis. Proceedings of the 58th annual meeting of the association for computational linguistics, 3211–20. Washington.
   Google Scholar
• Nils, R., and G. Iryna. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-Networks. Proceedings of the 2019 Conference on Empirical Methods in Natural Language, 3980–90. Hong Kong.
   Google Scholar
• Paul, N., V. Maarten, and R. Mihai. 2016. Learning text similarity with siamese recurrent networks. Proceedings of the 1st Workshop on Representation Learning for NLP, 148–57. Berlin.
   Google Scholar
• Pengcheng, H., G. Jianfeng, and C. Weizhu. 2023. DeBERTav3: Improving DeBERTa using ELECTRA-Style pre-training with gradient-disentangled embedding sharing. The Eleventh International Conference on Learning Representations, Kigali.
   Google Scholar
• Raffel, C., N. Shazeer, A. Roberts, K. Lee, S. Narang, Matena, M., Zhou, Y., Li, W. and Liu, P.J. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research 21 (1):5485–551.
   Google Scholar
• Sanh, V., L. Debut, J. Chaumond, and T. Wolf. 2019 Mar 1. DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter. arXiv Preprint arXiv 1910:01108.
   Google Scholar
• Shuai, Z., W. Lijie, S. Ke, and X. Xinyan. 2020 Sep 3. A practical Chinese dependency parser based on a large-scale dataset. arXiv Preprint arXiv 2009:00901.
   Google Scholar
• Song, C., and L. Hai. 2022. BERT-Log: Anomaly detection for system logs based on pre-trained language model. Applied Artificial Intelligence 36 (1). doi:10.1080/08839514.2022.2145642.
   Web of Science ®Google Scholar
• Tianyu, G., Y. Xingcheng, and C. Danqi. 2021. SimCSE - simple contrastive learning of sentence embeddings. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 6894–910. Punta Cana.
   Google Scholar
• Velikovi, P., G. Cucurull, A. Casanova, A. Romero, P. Liò,and Bengio, Y. 2017 Feb 4. Graph attention networks. arXiv Preprint axXiv 1710:10903.
   Google Scholar
• Weidong, Z., L. Xiaotong, J. Jun, and X. Rongchang. 2022. Re-LSTM: A long short-term memory network text similarity algorithm based on weighted word embedding. Connection Science 34 (1):2652–70. doi:10.1080/09540091.2022.2140122.
   Web of Science ®Google Scholar
• Wenjuan, L., S. Zhengyan, W. Subo, Z. Shunxiang, Z. Guangli, and L. Chen. 2024. PS-GCN: Psycholinguistic graph and sentiment semantic fused graph convolutional networks for personality detection. Connection Science 36 (1). doi:10.1080/09540091.2023.2295820.
   Web of Science ®Google Scholar
• Yan, K., P. Bin, K. Yongqi, Y. Yun, C. Jianguo, and X. Xie. 2024. Two-stage perceptual quality oriented rate control algorithm for HEVC. ACM Transactions on Multimedia Computing, Communications and Applications 20 (5):1–20. doi:10.1145/3636510.
   Google Scholar
• Yang, Z., Z. Dai, Y. Yang, G. Jaime, Salakhutdinov, R.R. and Le, Q.V. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. Proceedings of the 33rd International Conference on Neural Information Processing Systems, 5753–63. Vancouver.
   Google Scholar
• Yangfan, L., C. Cen, D. Mingxing, Z. Zeng, and L. Kenli. 2021. Attention-aware encoder–decoder neural networks for heterogeneous graphs of things. IEEE Transactions on Industrial Informatics 17 (4):2890–98. doi:10.1109/TII.2020.3025592.
   Web of Science ®Google Scholar
• Yifan, P., Y. Shankai, and L. Zhiyong. 2019. Transfer learning in biomedical natural language processing: An evaluation of BERT and ELMo on ten benchmarking datasets. Proceedings of the 18th BioNLP Workshop and Shared Task, 58–65. Florence.
   Google Scholar
• Yuanhe, T., C. Guimin, S. Yan, and W. Xiang. 2021. Dependency-driven relation extraction with attentive graph convolutional networks. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, 4458–71. Bangkok.
   Google Scholar
• Yung-Sung, C., D. Rumen, L. Hongyin, Z. Yang, C. Shiyu, Soljačić, M., Li, S.W., Yih, W.T., Kim, Y. and Glass, J., et al. 2022. DiffCSE: Difference-based contrastive learning for sentence embeddings. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 4207–18. Seattle.
   Google Scholar
• Yu, S., W. Shuohuan, L. Yukun, F. Shikun, T. Hao,Wu, H. and Wang, H. 2020. Ernie 2.0: A continual pre-training framework for language understanding. Proceedings of the AAAI conference on artificial intelligence, 8968–75. New York.
   Google Scholar
• Zeming, L. A. Halil, R. Roshan, H. Brian, Z. Zhongkai, L. Wenting, S. Nikita, V. Robert, K. Ori, S. Yaniv, et al. 2023. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science 379 (6637):1123–30.
   Web of Science ®Google Scholar
• Zhaorui, T., Y. Xi, Y. Zihan, W. Qiufeng, Y. Yuyao, A. Nguyen, and K. Huang. 2023. Semantic similarity distance: Towards better text-image consistency metric in text-to-image generation. Pattern Recognit 144:109883–109883. doi:10.1016/j.patcog.2023.109883.
   Web of Science ®Google Scholar
• Zhe, Q., W. Zhi-Jie, L. Yuquan, Y. Bin, L. Kenli, and J. Yin. 2019. An efficient framework for sentence similarity modeling. IEEE/ACM Transactions on Audio, Speech, and Language Processing 27 (4):853–65. doi:10.1109/TASLP.2019.2899494.
   Web of Science ®Google Scholar
• Zhiguo, W., M. Haitao, and I. Abraham. 2016 Feb 23. Sentence similarity learning by lexical decomposition and composition. arXiv Preprint axXiv 1602:07019.
   Google Scholar