US 11,983,626 B2
Method and apparatus for improving quality of attention-based sequence-to-sequence model
Min-Joong Lee, Suwon-si (KR)
Assigned to Samsung Electronics Co., Ltd., Suwon-si (KR)
Filed by SAMSUNG ELECTRONICS CO., LTD., Suwon-si (KR)
Filed on Dec. 2, 2020, as Appl. No. 17/109,490.
Claims priority of application No. 10-2020-0062450 (KR), filed on May 25, 2020.
Prior Publication US 2021/0366501 A1, Nov. 25, 2021
Int. Cl. G06N 3/02 (2006.01); G06N 3/04 (2023.01); G06N 3/045 (2023.01); G06N 3/08 (2023.01); G10L 15/16 (2006.01)
CPC G06N 3/08 (2013.01) [G06N 3/02 (2013.01); G06N 3/04 (2013.01); G06N 3/045 (2023.01); G10L 15/16 (2013.01)] 40 Claims
OG exemplary drawing
 
1. A method of improving the quality of an attention-based sequence-to-sequence model, the method comprising:
determining an output sequence corresponding to an input sequence based on an attention-based sequence-to-sequence model;
selecting at least one target attention head from among a plurality of attention heads each configured to generate a respective attention weight matrix;
detecting at least one error output token among output tokens constituting the output sequence based on the target attention head; and
correcting the output sequence based on the at least one error output token.