Image Caption: Image Caption using Deep learning - Brossura

Kannaiyan, Kanchana; R, Meenatchi

Brossura

ISBN 10: 6207647602 ISBN 13: 9786207647606

Casa editrice: LAP LAMBERT Academic Publishing, 2024

Riassunto

Image captioning with audio has emerged as a challenging yet promising task in the field of deep learning. This paper proposes a novel approach to address this task by integrating convolutional neural networks (CNNs) for image feature extraction and recurrent neural networks (RNNs) for sequential audio analysis. Specifically, we leverage pre-trained CNNs such as VGG to extract visual features from images and employ spectrogram representations coupled with RNNs such as LSTM or GRU to process audio inputs. Our proposed model based not only on their visual content but also on accompanying audio cues. We evaluate the performance of our model on benchmark datasets and demonstrate its effectiveness in generating coherent and contextually relevant captions for images with corresponding audio inputs. Additionally, we conduct tablation studies to analyze the contribution of each modality to the overall captioning performance, our results show that the fusion of visual and auditory modalities significantly improves captioning quality compared to using either modality in isolation.

Dettagli dell'edizione

• Editore: LAP LAMBERT Academic Publishing
• Data di pubblicazione: 2024
• Lingua: Inglese
• ISBN 10: 6207647602
• ISBN 13: 9786207647606
• Rilegatura: Copertina flessibile
• Numero di pagine: 64
• Contatto del produttore: non disponibile
• Persona responsabile: KreGa Onlinehandel GmbH; 4950000000000; info@krega-onlinehandel.de; ; Birkenweg 31; Brakel; 33034; Germania