| CPC G10L 25/30 (2013.01) [G06F 40/284 (2020.01); G06N 3/02 (2013.01); G10L 13/08 (2013.01)] | 13 Claims |
|
1. An electronic device comprising:
a memory storing a neural network model; and
a processor configured to input, to the neural network model, input data to obtain output data,
wherein, based on comparison between first output data based on input first modality and second output data based on input second modality, in response to the second modality being input, to output the first modality corresponding to the first output data based on the neural network model,
wherein the second modality comprises at least one masking element,
wherein the first modality comprises a first text and the second modality comprises a first audio signal, and
wherein the neural network model is configured to:
output a second audio signal corresponding to the first text and a second text corresponding to the first audio signal with the first text composed of a plurality of tokenized text elements and the first audio signal in which at least one of segmented plurality of elements is masked as input data, and
based on a first audio signal comprising the at least one masking element being input based on the comparison of the second audio signal and the second text, output a first text corresponding to the second audio signal.
|