Evaluation of a Segmental Durations Model for TTS

Transcription

1 Speech NLP Session Evaluation of a Segmental Durations Model for TTS João Paulo Teixeira, Diamantino Freitas* Instituto Politécnico de Bragança *Faculdade de Engenharia da Universidade do Porto

2 Overview Objectives Labelled Database for European Portuguese FEUP/IPB-DB Pre-processing Syllable division Intonation groups Set of features Model Objective Evaluation Subjective Evaluation Perceptual Test Audio examples Conclusion and future work

3 Objectives Prosody model for FEUPTTS Input features automatically extracted from text FEUP/IPB-DB were used for analysis/training. Extract durations for 44 types of segments of Portuguese set of phonemes: 9 oral vowels - 6 stop consonants nasal semi-vowels - 6 occlusive part of stop cons. semi-vowels - 3 nasal consonants 5 nasal vowels - 5 lateral consonants - 6 fricative consonants

4 Corpus FEUP/IPB-DB Texts were extracted from Portuguese newspapers containing all types of sentences. Read speech from professional male speaker in a regular broadcast studio. Speech ratio of. phonemes/sec. Manually labelled DB in 3 levels: Segments of phonemes and pauses tonic syllable in the word Word boundaries Phrase boundaries (including all punctuation marks) Training set - 6 tracks total of 8 minutes of speech Test set track total of 4 minutes of speech. Example can be found in

5 Relative frequency of segments in FEUP/IPB-DB (%) a 6 E i O o u j w j~ w~6~ e~ i~o~ u~ p!p t!t k!k b!b d!d g!g m n J l l* L r R v f z s S Z

6 Relative frequency of segments (%) Training set Test set a 6 E i O o u j w j~ w~ 6~ e~ i~ o~ u~ p!p t!t k!k b!b d!d g!g m n J l l* L r R v f z s S Z

7 Pre-processing Syllabification Applied to phonemes sequences error =.89% Applied to text error =.6% Intonation groups Simple rules without knowledge of syntax and semantic: Group of words with more than syllables in total. Group never ends with words of less than 3 phonemes. Phrase marks are always group boundaries. Last tonic syllable is the group accent. Example: É a guerra declarada, entre a Distrital socialista de Portalegre, liderada por Ceia da Silva, e a concelhia de Elvas, presidida por Rondão de Almeida.

8 Set of Features Feature - Codification # nodes Segment Identity of Distance to tonic syllable coded in 5 levels Previous segment (-) of Next segment (+) of Next segment (+) of 4 4 Next segment (+3) of Distance to next pause number of segments Position in phrase from beginning and end of phrase Position in group from beginning and end of group Type of vowel length in syllable coded in 5 levels Length of the accent group number of syllables and phonemes Position of accent group in sentence first; other; last. 3 Suppression or non-suppression of last vowel yes/not Type of syllable coded in 9 levels Type of previous syllable coded in 9 levels Type of vowel in previous syllable - coded in 5 levels Type of vowel in next syllable coded in 5 levels Correlation r. to to.3.5 to.8.8 to.4.5 to...4 to.4.3 to.5..3 to.5. to

9 Neural Network Model Architecture p p Σ p 3... Σ Σ Σ b, b, b, Σ b 3, b,3 Σ b, Σ d b,4 p n

10 General Results Equation Training set Test set i d i σ =, d = e e, e = x y N r V = = XY, XY,, VXY, σx. σy δ = i i i i i i N e i ( xi x).( yi y) i N 9,85 ms.834 4,7 ms 9,46 ms.839 4,3 ms

11 Target and Predicted Durations in Test set 5 R =.839 Best Linear Fit: A = (.68) T + (6.9) Data Points Best Linear Fit A = T 5 A T

12 e (ms) Normal Probability Plot Probability e (ms)

13 Histogram of real and predicted durations for vowel [a] 5 a - Durações reais ms a - Durações estimadas ms

14 Perceptual Evaluation 5 Paragraphs Blind test with 3 stimulus of each paragraph (original, model, average), presented in random order. 9 subjects Evaluation of rhythmic characteristics in a scale of 5.

15 Perceptual Test Score's by Subjects and Paragraph Original Model Average Original Model Average Average 3 Average Subjects Paragraph

16 Perceptual Test General Score's 5 Values Real Model Average

17 5 Conhece a situação na pele. Aprendeu-a na idade em que se aprende e se não esquece. Original Duration Predicted Duration 5 Average Duration

18 Audio Examples Conhece a situação na pele. Aprendeu-a na idade em que se aprende e se não esquece. Original Psola Manipulation of predicted durations σ =.8 ms; corr. coef. =.9 Psola Manipulation of average durations σ = 33.4 ms; Mas, que igualdade perante a lei? Que igualdade quando para muitos a justiça é praticamente inacessível? Como podem esses reclamar o cumprimento da lei, sem dinheiro para pagar a bons advogados e os elevados custos de um processo? Original Psola Manipulation of predicted durations σ = 9.8 ms; corr. coef. =.84

19 Conclusions Neural Network model to predict segmental durations. Objective evaluation gives an r =.84 and σ = 9.5 ms. Subjective evaluation gives a score of 4. against 4.3 for original and 3.53 for the average durations. More Linguistic information may possible improve the model. Considering the applicability to TTS systems, differences less than cycle ( ms) are irrelevant in voiced sounds. Major errors happen in very long sounds.