The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. ex. Some numerals are expressed as "XNUMX".
Copyrights notice
The original paper is in English. Non-English content has been machine-translated and may contain typographical errors or mistranslations. Copyrights notice
Penjanaan bukan autoregresif telah menarik lebih banyak perhatian kerana kelajuan penyahkodannya yang pantas. Objektif penjajaran terpendam, seperti CTC, direka bentuk untuk menangkap penjajaran monotonik antara token yang diramalkan dan token keluaran, yang telah digunakan untuk terjemahan mesin dan ringkasan ayat. Walau bagaimanapun, eksperimen awal kami mendedahkan bahawa CTC berprestasi buruk pada ringkasan abstrak dokumen, di mana nisbah mampatan tinggi antara input dan output terlibat. Untuk menangani isu ini, kami menjalankan analisis teori dan mencadangkan Penjajaran Terpendam Hierarki (HLA). Idea asas ialah proses penjajaran dua langkah: kami mula-mula menyelaraskan ayat dalam input dan output, dan seterusnya memperoleh penjajaran tahap token menggunakan CTC berdasarkan ayat sejajar. Kami menilai keberkesanan pendekatan cadangan kami pada dua set data yang digunakan secara meluas XSUM dan CNNDM. Keputusan menunjukkan bahawa kaedah yang dicadangkan kami mempamerkan kebolehskalaan yang luar biasa walaupun ketika berurusan dengan nisbah mampatan yang tinggi.
Wang XU
Harbin Institute of Technology
Yongliang MA
Beijing Langboat Technology Co., Ltd.
Kehai CHEN
Harbin Institute of Technology
Ming ZHOU
Beijing Langboat Technology Co., Ltd.
Muyun YANG
Harbin Institute of Technology
Tiejun ZHAO
Harbin Institute of Technology
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Salinan
Wang XU, Yongliang MA, Kehai CHEN, Ming ZHOU, Muyun YANG, Tiejun ZHAO, "Hierarchical Latent Alignment for Non-Autoregressive Generation under High Compression Ratio" in IEICE TRANSACTIONS on Information,
vol. E107-D, no. 3, pp. 411-419, March 2024, doi: 10.1587/transinf.2023EDP7111.
Abstract: Non-autoregressive generation has attracted more and more attention due to its fast decoding speed. Latent alignment objectives, such as CTC, are designed to capture the monotonic alignments between the predicted and output tokens, which have been used for machine translation and sentence summarization. However, our preliminary experiments revealed that CTC performs poorly on document abstractive summarization, where a high compression ratio between the input and output is involved. To address this issue, we conduct a theoretical analysis and propose Hierarchical Latent Alignment (HLA). The basic idea is a two-step alignment process: we first align the sentences in the input and output, and subsequently derive token-level alignment using CTC based on aligned sentences. We evaluate the effectiveness of our proposed approach on two widely used datasets XSUM and CNNDM. The results indicate that our proposed method exhibits remarkable scalability even when dealing with high compression ratios.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2023EDP7111/_p
Salinan
@ARTICLE{e107-d_3_411,
author={Wang XU, Yongliang MA, Kehai CHEN, Ming ZHOU, Muyun YANG, Tiejun ZHAO, },
journal={IEICE TRANSACTIONS on Information},
title={Hierarchical Latent Alignment for Non-Autoregressive Generation under High Compression Ratio},
year={2024},
volume={E107-D},
number={3},
pages={411-419},
abstract={Non-autoregressive generation has attracted more and more attention due to its fast decoding speed. Latent alignment objectives, such as CTC, are designed to capture the monotonic alignments between the predicted and output tokens, which have been used for machine translation and sentence summarization. However, our preliminary experiments revealed that CTC performs poorly on document abstractive summarization, where a high compression ratio between the input and output is involved. To address this issue, we conduct a theoretical analysis and propose Hierarchical Latent Alignment (HLA). The basic idea is a two-step alignment process: we first align the sentences in the input and output, and subsequently derive token-level alignment using CTC based on aligned sentences. We evaluate the effectiveness of our proposed approach on two widely used datasets XSUM and CNNDM. The results indicate that our proposed method exhibits remarkable scalability even when dealing with high compression ratios.},
keywords={},
doi={10.1587/transinf.2023EDP7111},
ISSN={1745-1361},
month={March},}
Salinan
TY - JOUR
TI - Hierarchical Latent Alignment for Non-Autoregressive Generation under High Compression Ratio
T2 - IEICE TRANSACTIONS on Information
SP - 411
EP - 419
AU - Wang XU
AU - Yongliang MA
AU - Kehai CHEN
AU - Ming ZHOU
AU - Muyun YANG
AU - Tiejun ZHAO
PY - 2024
DO - 10.1587/transinf.2023EDP7111
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E107-D
IS - 3
JA - IEICE TRANSACTIONS on Information
Y1 - March 2024
AB - Non-autoregressive generation has attracted more and more attention due to its fast decoding speed. Latent alignment objectives, such as CTC, are designed to capture the monotonic alignments between the predicted and output tokens, which have been used for machine translation and sentence summarization. However, our preliminary experiments revealed that CTC performs poorly on document abstractive summarization, where a high compression ratio between the input and output is involved. To address this issue, we conduct a theoretical analysis and propose Hierarchical Latent Alignment (HLA). The basic idea is a two-step alignment process: we first align the sentences in the input and output, and subsequently derive token-level alignment using CTC based on aligned sentences. We evaluate the effectiveness of our proposed approach on two widely used datasets XSUM and CNNDM. The results indicate that our proposed method exhibits remarkable scalability even when dealing with high compression ratios.
ER -