Company
Date Published
Author
Caroline Dockes
Word count
979
Language
English
Hacker News points
None

Summary

The text discusses the challenges and successes of implementing subword-level vocabulary in hybrid Automatic Speech Recognition (ASR) systems for English and German. The authors explore how using a word-level vocabulary is not feasible due to the constant evolution of language, and instead propose moving to a subword-level approach, which recognizes word pieces rather than entire words. They use Byte-Pair Encoding (BPE) as their tokenization algorithm and report promising results in German, where models trained with subwords perform well and recognize compound words that were previously outside the vocabulary. However, experiments in English show disappointing results, likely due to issues with long-range dependencies, word delimiters, and pronunciation, which need to be addressed in future research. Despite these challenges, the authors believe that this approach can have benefits for languages like German, where it has already shown promise.