Pointer Generator dan Coverage Weighting untuk Memperbaiki Peringkasan Abstraktif

Authors

  • Agna Silpi Alpiani Telkom University
  • Suyanto Suyanto Telkom University

DOI:

https://doi.org/10.34818/INDOJC.2019.4.2.341

Abstract

Model Long Short – Term Memory (LSTM) sequence-to-sequence telah banyak digunakan untuk menyelesaikan tantangan dalam peringkasan teks. Namun, model ini masih memiliki dua masalah utama yaitu, kemunculan kata diluar kosakata atau sering disebut out-of-vocabulary (OOV), dan perulangan kata. Pada makalah ini, pointer generator dan coverage weighting diterapkan untuk mengatasi dia masalah tersebut. Dimulai dengan model sequence-to-sequence dasar. Kemudian kami kembangkan dengan attention mechanism yang telah ditambahkan coverage weigting pada perhitungannya untuk mengurangi terjadinya perulangan kata, dan mengganti encoder menjadi bi-directional LSTM. Setelah itu kami mengimplementasikan pointer generator yang dapat  menunjuk kembali ke kata dalam teks sumber dan menghasilkan kata jika bertemu dengan kata OOV. Menggunakan dataset artikel berita bahasa Inggris CNN/Daily Mail dan metrik evaluasi ROUGE score, model kami menghasilkan hasil yang mirip dengan ringkasan teks sumber.

Downloads

Download data is not yet available.

References

Kågebäck, M., Mogren, O., Tahmasebi, N., & Dubhashi, D. (2014). Extractive summarization using continuous vector space models. In Proceedings of the 2nd Workshop on Continuous Vector Space Models and their Compositionality (CVSC) (pp. 31-39).

Pachantouris, G. (2005). GreekSum–A Greek Text Summarizer. Word Journal of the International Linguistic Association, 1-45.

Jing, H. (2000). Sentence reduction for automatic text summarization. In Sixth Applied Natural Language Processing Conference.

Cheung, J. C. K., & Penn, G. (2014). Unsupervised sentence enhancement for automatic summarization. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 775-786).

Rush, A. M., Chopra, S., & Weston, J. (2015). A neural attention model for abstractive sentence summarization. arXiv preprint arXiv:1509.00685.

Chopra, S., Auli, M., & Rush, A. M. (2016). Abstractive sentence summarization with attentive recurrent neural networks. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 93-98).

Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473.

Vinyals, O., Fortunato, M., & Jaitly, N. (2015). Pointer networks. In Advances in Neural Information Processing Systems (pp. 2692-2700).

Gulcehre, C., Ahn, S., Nallapati, R., Zhou, B., & Bengio, Y. (2016). Pointing the unknown words. arXiv preprint arXiv:1603.08148.

Merity, S., Xiong, C., Bradbury, J., & Socher, R. (2016). Pointer sentinel mixture models. arXiv preprint arXiv:1609.07843.

Gu, J., Lu, Z., Li, H., & Li, V. O. (2016). Incorporating copying mechanism in sequence-to-sequence learning. arXiv preprint arXiv:1603.06393.

Miao, Y., & Blunsom, P. (2016). Language as a latent variable: Discrete generative models for sentence compression. arXiv preprint arXiv:1609.07317.

Nallapati, R., Zhou, B., Gulcehre, C., & Xiang, B. (2016). Abstractive text summarization using sequence-to-sequence rnns and beyond. arXiv preprint arXiv:1602.06023.

See, A., Liu, P. J., & Manning, C. D. (2017). Get to the point: Summarization with pointer-generator networks. arXiv preprint arXiv:1704.04368.

Tu, Z., Lu, Z., Liu, Y., Liu, X., & Li, H. (2016). Modeling coverage for neural machine translation. arXiv preprint arXiv:1601.04811.

Chen, Q., Zhu, X. D., Ling, Z. H., Wei, S., & Jiang, H. (2016, July). Distraction-Based Neural Networks for Modeling Document. In IJCAI (pp. 2754-2760).

Hermann, K. M., Kocisky, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., & Blunsom, P. (2015). Teaching machines to read and comprehend. In Advances in neural information processing systems (pp. 1693-1701).

Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.

Lin, C. Y. (2004). Rouge: A package for automatic evaluation of summaries. Text Summarization Branches Out.

Downloads

Additional Files

Published

2019-09-09

How to Cite

Alpiani, A. S., & Suyanto, S. (2019). Pointer Generator dan Coverage Weighting untuk Memperbaiki Peringkasan Abstraktif. Indonesian Journal on Computing (Indo-JC), 4(2), 169–176. https://doi.org/10.34818/INDOJC.2019.4.2.341

Issue

Vol. 4 No. 2 (2019): September, 2019

Section

Computer Science

License

  • Manuscript submitted to IndoJC has to be an original work of the author(s), contains no element of plagiarism, and has never been published or is not being considered for publication in other journals. 
  • Copyright on any article is retained by the author(s). Regarding copyright transfers please see below.
  • Authors grant IndoJC a license to publish the article and identify itself as the original publisher.
  • Authors grant IndoJC commercial rights to produce hardcopy volumes of the journal for sale to libraries and individuals.
  • Authors grant any third party the right to use the article freely as long as its original authors and citation details are identified.
  • The article and any associated published material is distributed under the Creative Commons Attribution 4.0License