Sequence Summarizing Neural Network for Speaker Adaptation

Abstract

In this paper we propose a DNN adaptation technique, where the i-vector extractor is replaced by a Sequence Summarizing Neural Network (SSNN). Similarly to i-vector extractor, the SSNN produces a “summary vector”, which represents acoustic summary of an utterance. Such vector is then appended to the input of main network, while both networks are trained together optimizing single loss function. Both the ivector and SSNN speaker adaptation methods are compared on AMI headset meeting data. The results show comparable performance of both techniques on FBANK system with frame-classification training. Moreover, appending both the i-vector and “summary vector” to the FBANK features leads to additional improvement comparable to the performance of fMLLR feature DNN system.