Article Sidebar

Download
PDF
Statistic
Read Counter : 0 Download : 0

Main Article Content

Abstract

Aspect-Based Sentiment Analysis (ABSA) often experiences a significant performance decline in cross-domain settings due to vocabulary variation and domain-specific aspect expressions. Although transformer-based models achieve strong in-domain performance, they primarily rely on contextual embeddings and often ignore the syntactic structures that remain consistent across domains. Existing methods rarely integrate structured decoding with adaptive syntactic fusion for robust aspect boundary detection. This paper proposes a syntactic-aware cross-domain ABSA framework based on DeBERTaV3 and BIO-CRF decoding to alleviate the above problems. The proposed model introduces part-of-speech and dependency-relation embeddings, in addition to contextual embeddings, and uses an attention-based model to dynamically fuse syntactic and semantic information at multiple levels. We further apply a Conditional Random Field (CRF) layer to enforce valid BIO transitions and enhance the consistency of multi-word aspect spans under domain shift. The model was evaluated in three English review domains: Restaurant, Laptop, and Device across six zero-shot cross-domain transfer settings (D→L, D→R, L→D, L→R, R→D, and R→L). Test results show consistent advances over robust transformer-based and prompt-based baselines. The proposed method yields F1 scores for aspect extraction between 0.72 and 0.81 and achieves sentiment classification accuracies between 74.32% and 85.19%. The best performance was achieved in the L→R transfer setting. Through paired bootstrap testing (p < 0.01), Statistical analysis confirms that the proposed model achieves significant improvements over baseline methods. The results demonstrate that incorporating explicit syntactic knowledge, adaptive feature fusion, and structured decoding substantially improves cross-domain generalization in fine-grained sentiment analysis.  

Keywords

Cross-domain ABSA DeBERTaV3 Bio-CRF Span extraction Domain adaptation

Article Details

How to Cite
Teki, U. A. ., & Ranjana, P. (2026). Cross-domain aspect-based sentiment analysis using DeBERTaV3 and bio-CRF: a syntactic-aware span extraction approach. Future Technology, 5(3), 224–239. Retrieved from https://fupubco.com/futech/article/view/961
Download Citation
Endnote/Zotero/Mendeley (RIS)
BibTeX
Bookmark and Share

References

  1. Chauhan, Ganpat & Nahta, Ravi & Meena, Yogesh & Gopalani, Dinesh. (2023). Aspect based sentiment analysis using deep learning approaches: A survey. Computer Science Review. 49. 100576. 10.1016/j.cosrev.2023.100576. DOI:10.1016/j.cosrev.2023.100576.
  2. Pontiki, Maria & Galanis, Dimitrios & Pavlopoulos, John & Papageorgiou, Harris & Androutsopoulos, Ion & Manandhar, Suresh. (2014). SemEval-2014 Task 4: Aspect Based Sentiment Analysis. Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014). 27-35. 10.3115/v1/S14-2004.
  3. Eneko Agirre, Carmen Banea, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Rada Mihalcea, German Rigau, and Janyce Wiebe. 2016. SemEval-2016 Task 1: Semantic Textual Similarity, Monolingual and Cross-Lingual Evaluation. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 497–511, San Diego, California. Association for Computational Linguistics. Doi: 10.18653/v1/S16-1081.
  4. John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira. 2001. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. In Proceedings of the Eighteenth International Conference on Machine Learning (ICML '01). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 282–289. http://dl.acm.org/citation.cfm?id=645530.655813.
  5. Huang, Z., Xu, W., & Yu, K. (2015). Bidirectional LSTM-CRF Models for Sequence Tagging. ArXiv, abs/1508.01991.
  6. Gowda, V. D., Suneel, S., Naidu, P. R., Ramanan, S. V., & Suneetha, S. (2024). Challenges and limitations of few-shot and zero-shot learning. In applying machine learning techniques to bioinformatics: few-shot and zero-shot methods (pp. 113-137). IGI Global Scientific Publishing. DOI: 10.4018/979-8-3693-1822-5.ch007.
  7. Devlin, Jacob & Chang, Ming-Wei & Lee, Kenton & Toutanova, Kristina. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. 10.48550/arXiv.1810.04805.
  8. Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, and Omer Levy. 2020. SpanBERT: Improving Pre-training by Representing and Predicting Spans. Transactions of the Association for Computational Linguistics, 8:64–77. DOI: 10.1162/tacl_a_00300
  9. Jan-David Krieger, Timo Spinde, Terry Ruas, Juhi Kulshrestha, and Bela Gipp. 2022. A domain-adaptive pre-training approach for language bias detection in news. In Proceedings of the 22nd ACM/IEEE Joint Conference on Digital Libraries (JCDL '22). Association for Computing Machinery, New York, NY, USA, Article 3, 1–7. https://doi.org/10.1145/3529372.3530932
  10. C. R. Bhat, J. Nandhini, N. S, B. Karthik, P. K. Lakineni and S. N. Taqui, "Evaluating the Robustness of Neural Networks Against Adversarial Perturbations," 2023 International Conference on Communication, Security and Artificial Intelligence (ICCSAI), Greater Noida, India, 2023, pp. 911-915, doi: 10.1109/ICCSAI59793.2023.10421282.
  11. He, Pengcheng & Liu, Xiaodong & Gao, Jianfeng & Chen, Weizhu. (2020). DeBERTa: Decoding-enhanced BERT with Disentangled Attention. 10.48550/arXiv.2006.03654.
  12. Kahil, A. M. Attention Is All You Need. https://doi.org/10.48550/ARXIV.1706.03762
  13. He, P., Gao, J., & Chen, W. (2021). DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing. ArXiv, abs/2111.09543.
  14. Y. Zhang, "Dependency Tree-Based Contrastive Attention for Aspect-Based Sentiment Analysis," 2025 8th International Conference on Advanced Algorithms and Control Engineering (ICAACE), Shanghai, China, 2025, pp. 2042-2046, doi: 10.1109/ICAACE65325.2025.11019293.
  15. Knoester, Joris & Frasincar, Flavius & Truşcǎ, Maria. (2022). Domain Adversarial Training for Aspect-Based Sentiment Analysis. 10.1007/978-3-031-20891-1_3.
  16. Ganesh, D., Rao, P.V.V., Reddy, N.S., Suneetha, S., Lakineni, P.K., Yoganandh, S. (2025). EXB_RNN: A Hybrid Ensemble Approach for Enhanced Aspect-Based Sentiment Analysis. In: Das, A.K., Nayak, J., Naik, B., Himabindu, M., Vimal, S., Pelusi, D. (eds) Computational Intelligence in Pattern Recognition. CIPR 2024. Lecture Notes in Networks and Systems, vol 1152. Springer, Singapore. https://doi.org/10.1007/978-981-97-8090-7_30.
  17. Bengong Yu and Zhaodi Fan. 2020. A comprehensive review of conditional random fields: variants, hybrids and applications. Artif. Intell. Rev. 53, 6 (Aug 2020), 4289–4333. https://doi.org/10.1007/s10462-019-09793-6
  18. Zang J. (2025). Leveraging BiLSTM-CRF and adversarial training for sentiment analysis in nature-based digital interventions: Enhancing mental well-being through MOOC platforms. Digital health, 11, 20552076251317345. https://doi.org/10.1177/20552076251317345
  19. T. K, S. Soman, L. Anitha, P. K. Lakineni, D. G. V and S. N. Taqui, "Leveraging Temporal Patterns with LSTMs Networks for Financial Forecasting: A New Stastical Machine Learning Approach," 2023 International Conference on Communication, Security and Artificial Intelligence (ICCSAI), Greater Noida, India, 2023, pp. 916-920, doi: 10.1109/ICCSAI59793.2023.10421705.
  20. Yequan Wang, Minlie Huang, Xiaoyan Zhu, and Li Zhao. 2016. Attention-based LSTM for Aspect-level Sentiment Classification. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 606–615, Austin, Texas. Association for Computational Linguistics. DOI:10.18653/v1/D16-1058.
  21. Dehong Ma, Sujian Li, Xiaodong Zhang, and Houfeng Wang. 2017. Interactive attention networks for aspect-level sentiment classification. In Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI'17). AAAI Press, 4068–4074. https://doi.org/10.24963/ijcai.2017/568.
  22. M. A. Saputra and E. B. Setiawan, "Aspect Based Sentiment Analysis Using Recurrent Neural Networks (RNN) on Social Media Twitter," 2023 International Conference on Data Science and Its Applications (ICoDSA), Bandung, Indonesia, 2023, pp. 265-270, doi: 10.1109/ICoDSA58501.2023.10276768.
  23. H. Nguyen and K. Shirai, "A Joint Model of Term Extraction and Polarity Classification for Aspect-based Sentiment Analysis," 2018 10th International Conference on Knowledge and Systems Engineering (KSE), Ho Chi Minh City, Vietnam, 2018, pp. 323-328, doi: 10.1109/KSE.2018.8573340.
  24. Alexander Rietzler, Sebastian Stabinger, Paul Opitz, and Stefan Engl. 2020. Adapt or Get Left Behind: Domain Adaptation through BERT Language Model Finetuning for Aspect-Target Sentiment Classification. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 4933–4941, Marseille, France. European Language Resources Association. https://aclanthology.org/2020.lrec-1.607/
  25. Kei Akuzawa, Yusuke Iwasawa, and Yutaka Matsuo. 2019. Adversarial Invariant Feature Learning with Accuracy Constraint for Domain Generalization. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2019, Würzburg, Germany, September 16–20, 2019, Proceedings, Part II. Springer-Verlag, Berlin, Heidelberg, 315–331. https://doi.org/10.1007/978-3-030-46147-8_19.
  26. Shang, J., Zhang, Y., Zhong, L. et al. Syntactic-Enhanced Multi-Task Learning Model for Aspect Sentiment Triplet Extraction. Data Sci. Eng. 10, 515–531 (2025). https://doi.org/10.1007/s41019-025-00289-8.
  27. Hao Zhang, Yu-N Cheah, Congqing He, and Feifan Yi. 2024. An Instruction Tuning-Based Contrastive Learning Framework for Aspect Sentiment Quad Prediction with Implicit Aspects and Opinions. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 7698–7714, Miami, Florida, USA. Association for Computational Linguistics. DOI:10.18653/v1/2024.findings-emnlp.453.
  28. Nakagawa, Tetsuji & Inui, Kentaro & Kurohashi, Sadao. (2010). Dependency Tree-based Sentiment Classification using CRFs with Hidden Variables.. NAACL-HLT. 786-794. https://aclanthology.org/N10-1120/.
  29. Zhang, F., Zheng, W. & Yang, Y. Graph Convolutional Network with Syntactic Dependency for Aspect-Based Sentiment Analysis. Int J Comput Intell Syst 17, 37 (2024). https://doi.org/10.1007/s44196-024-00419-6
  30. Qizhi Zhao, Zan Mo, and Mengting Fan. 2023. POS-ATAEPE-BiLSTM: an aspect-based sentiment analysis algorithm considering part-of-speech embedding. Applied Intelligence 53, 22 (Nov 2023), 27440–27458. https://doi.org/10.1007/s10489-023-04952-3.
  31. Xuezhe Ma and Eduard Hovy. 2016. End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1064–1074, Berlin, Germany. Association for Computational Linguistics. Doi: 10.18653/v1/P16-1101.
  32. You Li, Yongdong Lin, Yuming Lin, Liang Chang, and Huibing Zhang. 2022. A span-sharing joint extraction framework for harvesting aspect sentiment triplets. Know.-Based Syst. 242, C (Apr 2022). https://doi.org/10.1016/j.knosys.2022.108366.
  33. Arora, J., Shivanka, Jain, A., Nikunj, Dahiya, S., Pandey, V.K. (2025). T5 Generator: An Aspect-Based Analysis of Sentiments. In: Virmani, D., Castillo, O., Balas, V.E., Elngar, A.A. (eds) Proceedings of International Conference on Generative AI, Cryptography and Predictive Analytics. ICGCPA 2024. Studies in Smart Technologies. Springer, Singapore. https://doi.org/10.1007/978-981-97-9132-3_12.
  34. Sun, Xinjie & Zhang, Kai & Liu, Qi & Bao, Meikai & Chen, Yanjiang. (2024). Harnessing domain insights: A prompt knowledge tuning method for aspect-based sentiment analysis. Knowledge-Based Systems. 298. 111975. 10.1016/j.knosys.2024.111975.
  35. Yue Deng, Wenxuan Zhang, Sinno Jialin Pan, and Lidong Bing. 2023. Bidirectional Generative Framework for Cross-domain Aspect-based Sentiment Analysis. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12272–12285, Toronto, Canada. Association for Computational Linguistics. Doi: 10.18653/v1/2023.acl-long.686.
  36. Chouikhi, H., Alsuhaibani, M., & Jarray, F. (2023). BERT-Based Joint Model for Aspect Term Extraction and Aspect Polarity Detection in Arabic Text. Electronics, 12(3), 515. https://doi.org/10.3390/electronics12030515
  37. Clark, Kevin & Luong, Minh-Thang & Le, Quoc & Manning, Christopher. (2020). ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators. 10.48550/arXiv.2003.10555.
  38. Maria Pontiki, Dimitris Galanis, Haris Papageorgiou, Suresh Manandhar, and Ion Androutsopoulos. 2015. SemEval-2015 Task 12: Aspect Based Sentiment Analysis. In Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 486–495, Denver, Colorado. Association for Computational Linguistics. Doi: 10.18653/v1/S15-2082.
  39. Eneko Agirre, Carmen Banea, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Rada Mihalcea, German Rigau, and Janyce Wiebe. 2016. SemEval-2016 Task 1: Semantic Textual Similarity, Monolingual and Cross-Lingual Evaluation. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 497–511, San Diego, California. Association for Computational Linguistics. Doi: 10.18653/v1/S16-1081.
  40. Alghamdi, Salem & Alhasawi, Yaser. (2024). Aspect-Based Sentiment Analysis in Smart Devices: A Comprehensive and Specialized Dataset. Data in Brief. 55. 110642. 10.1016/j.dib.2024.110642.
  41. Phillip Howard, Arden Ma, Vasudev Lal, Ana Paula Simoes, Daniel Korat, Oren Pereg, Moshe Wasserblat, and Gadi Singer. 2022. Cross-Domain Aspect Extraction using Transformers Augmented with Knowledge Graphs. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management (CIKM '22). Association for Computing Machinery, New York, NY, USA, 780–790. https://doi.org/10.1145/3511808.3557275
  42. Wenya Wang, Sinno Jialin Pan; Syntactically Meaningful and Transferable Recursive Neural Networks for Aspect and Opinion Extraction. Computational Linguistics 2019; 45 (4): 705–736. doi: https://doi.org/10.1162/coli_a_00362