Image Caption: Image Caption using Deep learning - Softcover

Kannaiyan, Kanchana; R, Meenatchi

Softcover

ISBN 10: 6207647602 ISBN 13: 9786207647606

Verlag: LAP LAMBERT Academic Publishing, 2024

Inhaltsangabe

Image captioning with audio has emerged as a challenging yet promising task in the field of deep learning. This paper proposes a novel approach to address this task by integrating convolutional neural networks (CNNs) for image feature extraction and recurrent neural networks (RNNs) for sequential audio analysis. Specifically, we leverage pre-trained CNNs such as VGG to extract visual features from images and employ spectrogram representations coupled with RNNs such as LSTM or GRU to process audio inputs. Our proposed model based not only on their visual content but also on accompanying audio cues. We evaluate the performance of our model on benchmark datasets and demonstrate its effectiveness in generating coherent and contextually relevant captions for images with corresponding audio inputs. Additionally, we conduct tablation studies to analyze the contribution of each modality to the overall captioning performance, our results show that the fusion of visual and auditory modalities significantly improves captioning quality compared to using either modality in isolation.

Über diesen Titel

Über die Autorin bzw. den Autor

Ich bin Frau K. Kanchana und arbeite als Assistenzprofessorin in der Abteilung für Informatik und Ingenieurwesen am Kathir College of Engineering. Ich interessiere mich für das Gebiet des maschinellen Lernens und des Deep Learning.

Details zur Ausgabe

• Verlag: LAP LAMBERT Academic Publishing
• Erscheinungsdatum: 2024
• Sprache: Englisch
• ISBN 10: 6207647602
• ISBN 13: 9786207647606
• Einband: Taschenbuch
• Anzahl der Seiten: 64
• Kontakt zum Hersteller: SIA OmniScriptum Publishing; customerservice@vdm-vsg.de; ; Brivibas gatve 197; Riga; 1039; Lettland