The Blame Game in Meeting Room ASR: An Analysis of Feature versus Model Errors in Noisy and Mismatched Conditions

TitleThe Blame Game in Meeting Room ASR: An Analysis of Feature versus Model Errors in Noisy and Mismatched Conditions
Publication TypeConference Paper
Year of Publication2013
AuthorsParthasarathi, S. Hari Krish, Chang S-Y., Cohen J., Morgan N., & Wegmann S.
Other Numbers3441
Abstract

Given a test waveform, state-of-the-art ASR systems extract a sequenceof MFCC features and decode them with a set of trainedHMMs. When this test data is clean, and it matches the conditionused for training the models, then there are few errors. While it isknown that ASR systems are brittle in noisy or mismatched conditions,there has been little work in quantitatively attributing the errorsto features or to models. This paper attributes the sources of these errorsin three conditions: (a) matched near-field, (b) matched far-field,and a (c) mismatched condition. We undertake a series of diagnosticanalyses employing the bootstrap method to probe a meeting roomASR system. Results show that when the conditions are matched(even if they are far-field), the model errors dominate; however, inmismatched conditions features are neither invariant nor separableand this causes as many errors as the model does.

Acknowledgment

This work was partially supported by funding provided to ICSI by the Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA) through the project "What’s wrong with ASR, and how can we fix it." All statements of fact, opinion or conclusions contained herein are those of the authors and should not be construed as representing the official views or policies of the IARPA, the ODNI or the U.S. Government.

URLhttps://www.icsi.berkeley.edu/pubs/speech/blamegame13.pdf
Bibliographic Notes

Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2013), Vancouver, Canada

Abbreviated Authors

S. H. Krishnan Parthasarathi, S.-Y. Chang, J. Cohen, N. Morgan, and S. Wegmann

ICSI Research Group

Speech

ICSI Publication Type

Article in conference proceedings