Adaptive Discounting of Implicit Language Models in RNN-Transducers

Unni, Vinit ; Khare, Shreya ; Mittal, Ashish ; Jyothi, Preethi ; Sarawagi, Sunita ; Bharadwaj, Samarth (2022) Adaptive Discounting of Implicit Language Models in RNN-Transducers In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[img] PDF
292kB

Official URL: http://doi.org/10.1109/ICASSP43922.2022.9747115

Related URL: http://dx.doi.org/10.1109/ICASSP43922.2022.9747115

Abstract

RNN-Transducer (RNN-T) models have become synonymous with streaming end-to-end ASR systems. While they perform competitively on a number of evaluation categories, rare words pose a serious challenge to RNN-T models. One main reason for the degradation in performance on rare words is that the language model (LM) internal to RNN-Ts can be-come overconfident and lead to hallucinated predictions that are acoustically inconsistent with the underlying speech. To address this issue, we propose a lightweight adaptive LM dis-counting technique ADAPTLMD, that can be used with any RNN-T architecture without requiring any external resources or additional parameters. ADAPTLMD uses a two-pronged approach: 1. Randomly mask the prediction network output to encourage the RNN-T to not be overly reliant on it’s outputs. 2. Dynamically choose when to discount the implicit LM (ILM) based on rarity of recently predicted tokens and divergence between ILM and implicit acoustic model (IAM) scores. Comparing ADAPTLMD to a competitive RNN-T baseline, we obtain up to 4% and 14% relative reductions in overall WER and rare word PER, respectively, on a conversational, code-mixed Hindi-English ASR task.

Item Type:Conference or Workshop Item (Paper)
Source:Copyright of this article belongs to IEEE
Keywords:Degradation;Adaptation models;Conferences;Predictive models;Signal processing;Acoustics;Task analysis
ID Code:128269
Deposited On:19 Oct 2022 03:55
Last Modified:15 Nov 2022 08:44

Repository Staff Only: item control page