×
Home Current Archive Editorial board
Instructions for papers
For Authors Aim & Scope Contact
Original scientific article

APPLICATION OF HYBRID & NOVEL DEEP LEARNING APPROACHES FOR MULTIMODAL SENTIMENT FUSION IN IMAGES & AUDIO ANALYSIS

By
Jayaprakash Vattikundala Orcid logo ,
Jayaprakash Vattikundala

Research Scholar, Department of ECM, Koneru Lakshmaiah Education Foundation , Vaddeswaram, Guntur, Andhra Pradesh , India

M. Siva Ganga Prasad Orcid logo
M. Siva Ganga Prasad

Professor & Coordinator (FED), Koneru Lakshmaiah Education Foundation , Vaddeswaram, Guntur, Andhra Pradesh , India

Abstract

The paper suggests a hybrid multimodal sentiment analysis (MSA) model that would enhance the accuracy of sentiment prediction through the combination of textual, auditory, and visual information. In most cases, the traditional sentiment analysis models have been challenged because of numerous overlapping features and poor fusion methods when using multimodal data. To overcome these problems, propose a supervised contrastive learning-based methodology that will improve data representation and exploit multimodal feature fusion. The technique includes pre-processing Twitter information by tokenization, stemming, and feature extraction, and classifying it with the help of a Particle Swarm Optimization-Deep Learning Modified Neural Network (PSO-DLBMNN). The experimental findings, assessed based on the measures of accuracy, precision, recall, and F1-score, demonstrate that the suggested model is superior to the traditional approaches to deep learning, such as Bi-LSTM and Bi-GRU. In particular, the PSO-DLBMNN model had an accuracy of 95.48, a precision of 96.57, a recall of 94.87, and an F1-score of 93.45, which is a substantial increase over the baseline models. These results indicate that the model is capable of completing multiple tasks of integrating multimodal data alongside solving the problem of redundancy and data noise. The suggested method gives a fresh outlook on improving sentiment analysis through enhancing multimodal feature fusion. To sum up, the model has the potential to be applied to real-time analysis in social media and human-computer interaction systems, and it provides information about how multimodal data can be used to enhance sentiment prediction and emotional perception.

References

1.
Ain QT, Ali M, Riaz A, Noureen A, Kamran M, Hayat B, Rehman AU. Sentiment analysis using deep learning techniques: a review. International Journal of Advanced Computer Science and Applications. 2017;8(6). 424-433.
2.
Prasad KR, Karanam SR, Ganesh D, Liyakat KK, Talasila V, Purushotham P. AI in public-private partnership for IT infrastructure development. The Journal of High Technology Management Research. 2024 May 1;35(1):100496.
3.
Rahman F. Scalable Safety-Constrained Learning Pipelines for Distributed Digital-Twin-Based Energy Optimization in Large-Scale Electric Mobility Systems. SECITS Journal of Scalable Distributed Computing and Pipeline Automation. 2026 Jan 10:1-8.
4.
Balakrishna N, Krishnan MB, Ganesh D. Hybrid Machine Learning Approaches for Predicting and Diagnosing Major Depressive Disorder. International Journal of Advanced Computer Science & Applications. 2024 Mar 1;15(3).
5.
Turukmane AV, Tangudu N, Sreedhar B, Ganesh D, Reddy PS, Batta U. An effective routing algorithm for load balancing in unstructured peer-to-peer networks. International Journal of Intelligent Systems and Applications in Engineering. 2023;12(7s):87-97.

Citation

This is an open access article distributed under the  Creative Commons Attribution Non-Commercial License (CC BY-NC) License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. 

Article metrics

Google scholar: See link

The statements, opinions and data contained in the journal are solely those of the individual authors and contributors and not of the publisher and the editor(s). We stay neutral with regard to jurisdictional claims in published maps and institutional affiliations.