Paper Accepted BY IEEE Transactions on Audio, Speech and Language Processing (IEEE TASLP)