Application of Text Summarization on Text-Based Generative Adversarial Networks
Keywords:
Generative adversarial networks, Text Summarization, BART transformer, DALL-E mini, generator, discriminatorAbstract
In this project, we wish to convert long textual inputs into summarised text chunks and generate images describing the summarized text. This project aims to cultivate a model that can generate true-to-life images from summarized textual input using GAN. GANs aim to estimate and recreate the possible spread of real-world data samples and produce new pictures based on this distribution. This project offers an automated summarised text-to-image synthesis for creating images from written descriptions. The written descriptions serve as the GAN generator's conditional intake. The first step in this synthesis is the use of Natural Language Processing to bring out keywords for summarizing. BART transformers are employed. This is then fed to the GAN network consisting of a generator and discriminator. This project used a pre-trained DALL-E mini model as the GAN architecture.
References
D. Berthelot, T. Schumm, and L. Metz, "Began: Boundary equilibrium generative adversarial networks," 2017, pp. 8.
K. Crowson, S. Biderman, D. Kornis, D. Stander, E. Hallahan, L. Castricato, and E. Raff, "Vqgan-clip: Open domain image generation and editing with natural language guidance," 2022, pp. 16.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, "Bert: Pre-training of deep bidirectional transformers for language understanding," 2018.
I. C. Education, "What are Neural Networks? " Internet: https://www.ibm.com/cloud/learn/neural-networks, 2020 [July 2022].
I. Goodfellow, " Generative adversarial networks," presented at NIPS on GANs, 2016.
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, "Generative adversarial nets," Advances in Neural Information Processing Systems (NIPS), Springer New York, 2014.
C. Gou, Y. Wu, K. Wang, F.-Y. Wang, and Q. Ji, "Learning-by-synthesis for accurate eye detection," 23rd International Conference on Pattern Recognition (ICPR). 2016.
J. Gui, Z. Sun, Y. Wen, D. Tao, and J. Ye, "A review on generative adversarial networks: Algorithms, theory, and applications," IEEE Transactions on Knowledge and Data Engineering pp.3313-3332, vol. 35 2020.
R. Huang, S. Zhang, T. Li, and R. He, "Beyond face rotation: Global and local perception GAN for photorealistic and identity preserving frontal view synthesis," Proc. of the IEEE International Conference on Computer Vision (ICCV). 2017
S. Ioffe and C. Szegedy, "Batch normalization: Accelerating deep network training by reducing internal covariate shift," International Conference on Machine Learning, 2015.
C. Ledig, L. Theis, F. Huszár, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, and Z. Wang, "Photo-realistic single image super-resolution using a generative adversarial network," Proc. IEEE Conference on Computer Vision and Pattern Recognition, 2017.
M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, and L. Zettlemoyer, "Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension," 2019.
R. Mihalcea and P. Tarau, "Textrank: Bringing order into text," Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, 2004.
R. Nallapati, B. Zhou, C. Gulcehre, and B. Xiang, "Abstractive text summarization using sequence-to-sequence RNNs and beyond," 2016.
D. Radev, E. Hovy, and K. McKeown, "Introduction to the special issue on summarization," Computational Linguistics, vol. 28, no. 4, pp. 399-408, 2002.
E. Santana and G. Hotz, "Learning a driving simulator," 2016.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, ?. Kaiser, and I. Polosukhin, "Attention is all you need," Advances in Neural Information Processing Systems, vol. 30, 2017.
K. Wang, C. Gou, Y. Duan, Y. Lin, X. Zheng, and F.-Y. Wang, "Generative adversarial networks: introduction and outlook," IEEE/CAA Journal of Automatica Sinica, vol. 4, no. 4, pp. 588-598, 2017.
Wikipedia, "PageRank," PageRank. [Online]. Internet: https://en.wikipedia.org/wiki/PageRank. 16 August 2014 [2022]
T. Xu, P. Zhang, Q. Huang, H. Zhang, Z. Gan, X. Huang, and X. He, "Attngan: Fine-grained text to image generation with attentional generative adversarial networks," Proc. IEEE Conference on Computer Vision and Pattern Recognition, 2018.
J. Zheng, Y. Cai, Y. Wu, and X. Shen, "Dynamic computation offloading for mobile cloud computing: A stochastic game-theoretic approach," IEEE Transactions on Mobile Computing, vol. 18, no. 4, pp. 771-786, 2018.
J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, "Unpaired image-to-image translation using cycle-consistent adversarial networks," Proc. IEEE International Conference on Computer Vision, 2017.
Mirza, M., & Osindero, S. “Conditional generative adversarial nets”, 2014.
Zhang, H., Sindagi, V., & Patel, V. M. “Image de-raining using a conditional generative adversarial network”. IEEE transactions on circuits and systems for video technology, 2019.
Larsen, A. B. L., Sønderby, S. K., Larochelle, H., & Winther, O. “Autoencoding beyond pixels using a learned similarity metric”. International conference on machine learning, 2016.
Donahue, J., Krähenbühl, P., & Darrell, T. “Adversarial feature learning.” 2016.
Metz, L., Poole, B., Pfau, D., & Sohl-Dickstein, J. “Unrolled generative adversarial networks”, 2016.
Mallasto, A., Montúfar, G., & Gerolin, A. “How well do WGANs estimate the Wasserstein”, 2019.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Muhammad Alli-Balogun
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Authors who submit papers with this journal agree to the following terms.
