Article Sidebar

Download
PDF
Statistic
Read Counter : 0 Download : 0

Main Article Content

Abstract

Tibetan art's ornate designs are widely used in social media promotions in the tourism industry, but predicting consumer reactions to ethnic advertising remains difficult. For some people, the elaborate ornamentation symbolizes cultural authenticity; for others, it seems staged. Most existing models analyze imagery and user commentary in isolation. Whether these two sources offer complementary predictors of audience response therefore remains unclear. Here we present a gated bidirectional cross-attention model to predict thirty-day audience acceptance of Tibetan-motif tourism advertisements on Xiaohongshu from visual representations and early twenty-four-hour comment sentiment. We first train the motif-recognition backbone and two interpretable descriptors (motif density and circular hue chromatic conformity) on 19,142 archived pattern images, then transfer them to 4,860 promotional posts using a month-stratified partitioning scheme. Acceptance is treated as a three-category ordinal target derived from engagement residuals adjusted for covariates. Sentiment measures calculated from 149,300 preliminary comments feed into an entropy-dependent adaptive gate that shifts the balance toward visual information as discussion threads become scarce. Our proposed model achieves 0.754 accuracy and 0.734 macro-F1 in experiments on hand-crafted, unimodal, concatenation, CLIP, and ViLT baselines. It beats standard late fusion by 3.5 and 3.7 percentage points on these two metrics and cuts ordinal Mean Absolute Error (MAE) by 12.3 percent relative (from 0.318 to 0.279). Cross-modal complementarity is validated by Accumulated Local Effects (ALE) and Ablation Analyses, with chromatic conformity and sentiment polarity increasing acceptance, while motif density exhibits an inverse-U relationship.

Keywords

Multimodal deep learning Cross-modal attention fusion Comment sentiment analysis Tibetan ornamental motifs Tourism advertising acceptance

Article Details

How to Cite
Hong, L., Nor Shahizan Ali, M., & Azul Mohamad Salleh, M. (2026). A multimodal deep learning model integrating visual features and early comment sentiment for predicting audience acceptance of Tibetan-motif tourism advertisements on Xiaohongshu. Future Technology, 6(1), 14–23. Retrieved from https://fupubco.com/futech/article/view/1207
Download Citation
Endnote/Zotero/Mendeley (RIS)
BibTeX
Bookmark and Share

References

  1. J. Ma and Z. Wu, “Platformized tourism circuits and urban space: Xiaohongshu, Wanghong places, and Dongshankou’s transformation,” Journal of Travel Research, vol. 65, no. 2, pp. 312-329, 2026. DOI: https://doi.org/10.1177/00472875251408445.
  2. Z. Wang, W.-J. Huang and B. Liu-Lastres, “Impact of user-generated travel posts on travel decisions: A comparative study on Weibo and Xiaohongshu,” Annals of Tourism Research Empirical Insights, vol. 3, no. 2, art. 100064, 2022. DOI: https://doi.org/10.1016/j.annale.2022.100064.
  3. K. He, X. Zhang, S. Ren and J. Sun, “Deep residual learning for image recognition,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016, pp. 770-778. DOI: https://doi.org/10.1109/CVPR.2016.90.
  4. A. Dosovitskiy et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” in Proc. Int. Conf. Learning Representations (ICLR), Vienna, Austria, 2021, pp. 1-21. DOI: https://doi.org/10.48550/arXiv.2010.11929.
  5. X. Peng et al., “Research on image recognition of intangible cultural heritage based on CNN and wireless network,” EURASIP Journal on Wireless Communications and Networking, vol. 2020, art. 240, 2020. DOI: https://doi.org/10.1186/s13638-020-01859-2.
  6. W. Wang, Q. Liu and Z. Chen, “Fine-grained recognition and segmentation of Tibetan thangka cultural heritage images based on multi-scale attention networks,” Heritage Science, vol. 11, art. 142, 2023. DOI: https://doi.org/10.1186/s40494-023-00984-x.
  7. J. Devlin, M.-W. Chang, K. Lee and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proc. NAACL-HLT, Minneapolis, MN, USA, 2019, pp. 4171-4186. DOI: https://doi.org/10.18653/v1/N19-1423.
  8. Y. Cui, W. Che, T. Liu, B. Qin and Z. Yang, “Pre-training with whole word masking for Chinese BERT,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 3504-3514, 2021. DOI: https://doi.org/10.1109/TASLP.2021.3124365.
  9. Y. Wen, Y. Liang and X. Zhu, “Sentiment analysis of hotel online reviews using the BERT model and ERNIE model—Data from China,” PLOS ONE, vol. 18, no. 3, art. e0275382, 2023. DOI: https://doi.org/10.1371/journal.pone.0275382.
  10. J. Liu, C. Wang and T. C. Zhang, “Exploring social media affordances in tourist destination image formation: A study on China’s rural tourism destination,” Tourism Management, vol. 101, art. 104843, 2024. DOI: https://doi.org/10.1016/j.tourman.2023.104843.
  11. Y. Dong, Y. Li, H.-Y. Hua and W. Li, “Perceived tourism authenticity on social media: The consistency of ethnic destination endorsers,” Tourism Management Perspectives, vol. 49, art. 101176, 2023. DOI: https://doi.org/10.1016/j.tmp.2023.101176.
  12. Y.-F. Liu, Y.-B. Zhu, H.-H. Wu and F. S. Li, “Are there any differences in the tourists’ perceived destination image between travel e-commerce platforms and social media platforms?,” Tourism Critiques: Practice and Theory, vol. 5, no. 2, pp. 229-253, 2024. DOI: https://doi.org/10.1108/TRC-05-2024-0018.
  13. M. Philp, J. Jacobson and E. Pancer, “Predicting social media engagement with computer vision: An examination of food marketing on Instagram,” Journal of Business Research, vol. 149, pp. 736-747, 2022. DOI: https://doi.org/10.1016/j.jbusres.2022.05.078.
  14. Y. Chen, H. Wang, S. Rao Hill and B. Li, “Consumer attitudes toward AI-generated ads: Appeal types, self-efficacy and AI’s social role,” Journal of Business Research, vol. 185, art. 114867, 2024. DOI: https://doi.org/10.1016/j.jbusres.2024.114867.
  15. I. C. Dang, C. M. C. Kwan, J. S. Jia and Y. Shi, “When words meet visuals: How content composition drives social media engagement for marketer-generated content,” Journal of Marketing Research, vol. 63, no. 1, pp. 118-139, 2026. DOI: https://doi.org/10.1177/00222437251373042.
  16. L. Liu, Y. Wang, Z. Fang and S. Wu, “The impact of verbal and visual content on consumer engagement in social media marketing,” Production and Operations Management, vol. 34, no. 8, pp. 2145-2164, 2025. DOI: https://doi.org/10.1177/10591478251349892.
  17. A. Gandhi, K. Adhvaryu, S. Poria, E. Cambria and A. Hussain, “Multimodal sentiment analysis: A systematic review of history, datasets, multimodal fusion methods, applications, challenges and future directions,” Information Fusion, vol. 91, pp. 424-444, 2023. DOI: https://doi.org/10.1016/j.inffus.2022.09.025.
  18. H. Li, Y. Lu and H. Zhu, “Multi-modal sentiment analysis based on image and text fusion based on cross-attention mechanism,” Electronics, vol. 13, no. 11, art. 2069, 2024. DOI: https://doi.org/10.3390/electronics13112069.
  19. L. Wang, J. Peng, C. Zheng, T. Zhao and L. Zhu, “A cross modal hierarchical fusion multimodal sentiment analysis method based on multi-task learning,” Information Processing & Management, vol. 61, no. 3, art. 103675, 2024. DOI: https://doi.org/10.1016/j.ipm.2024.103675.
  20. A. Radford et al., “Learning transferable visual models from natural language supervision,” in Proc. Int. Conf. Machine Learning (ICML), vol. 139, 2021, pp. 8748-8763. [Online]. Available: https://proceedings.mlr.press/v139/radford21a.html.
  21. W. Kim, B. Son and I. Kim, “ViLT: Vision-and-language transformer without convolution or region supervision,” in Proc. Int. Conf. Machine Learning (ICML), vol. 139, 2021, pp. 5583-5594. [Online]. Available: https://proceedings.mlr.press/v139/kim21k.html.
  22. B. Wu et al., “Predicting popularity of social media posts via multimodal hierarchical attention networks,” IEEE Transactions on Multimedia, vol. 25, pp. 4112-4125, 2023. DOI: https://doi.org/10.1109/TMM.2022.3171092.
  23. W. Cao, V. Mirjalili and S. Raschka, “Rank consistent ordinal regression for neural networks with application to age estimation,” Pattern Recognition Letters, vol. 140, pp. 325-331, 2020. DOI: https://doi.org/10.1016/j.patrec.2020.11.008.
  24. D. W. Apley and J. Zhu, “Visualizing the effects of predictor variables in black box supervised learning models,” Journal of the Royal Statistical Society Series B: Statistical Methodology, vol. 82, no. 4, pp. 1059-1086, 2020. DOI: https://doi.org/10.1111/rssb.12377.
  25. A. Vaswani et al., “Attention is all you need,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 30, Long Beach, CA, USA, 2017, pp. 5998-6008. DOI: https://doi.org/10.48550/arXiv.1706.03762.
  26. H. Talebi and P. Milanfar, “NIMA: Neural image assessment,” IEEE Transactions on Image Processing, vol. 27, no. 8, pp. 3998-4011, 2018. DOI: https://doi.org/10.1109/TIP.2018.2831899.