Postech English Speech Assessment and Assistant (PESAA)
- Demo Video
- Contact
Postech English Speech Assessment and Assistant (PESSA) is a sort of computer assisted pronunciation training system which makes the most use of computer to improve English learners' oral proficiency. PESAA system is designed to present intelligent feedback as well as detailed assessment. To our definition, the intelligent feedback is user and instance specific feedback under pedagogical consideration. We think finding error position and type from learners' input utterances with low false alarm rate is the first step to realize the intelligent feedback and are mainly working on users' variant/error simulation.
Intonation:It is hard to simulate English intonation because the intonation has a high degree of freedom (DOF) according to utterance circumstances. A generation method for English intonation is to utilize the ToBI-based prosody components. Especially, we chose prosodic components with relatively lower DOF: pitch accents, phrase accents and boundary tones. We have already researched the prediction and detection models for pitch accents, phrase accents and boundary tones as well as the feedback model based on those models.
Sentence Stress Error Detection:A simulator for sentence stress errors is not currently necessary because the sentence stress is not given by ASR decoder, and thus we need to build a baseline sentence stress detector. Our design of stress detector is similar to that of ASR, but it will produce a sequence of binary labels indicating stressed/un-stressed instead of pronunciation. We are also developing a sentence stress prediction method based on well formed knowledge which can produce desired sentence stress positions for given text/transcription. The stress error can be found by comparing the two results: predicted and detected stress positions.
Vowel Reduction:The stress patterns play a significant role in communication in English whereas the stress patterns carry no information in some other languages including Korean. For that reason, many Korean learners of English do not seriously consider the stress patterns, i.e., the Korean learners seldom reduce vowels and it results in every vowel sounds like stressed. However, in American English non-stressed vowels reduced to a special sound schwa. Schwa sound is short and neutral, that usually appears in American English. Schwa sound is most common vowel because the non-stressed vowels. Thus, reducing un-stressed vowels into schwa pronunciation is very important in American English speaking and its characteristic is deeply related to stress patterns. We are developing a vowel reduction prediction and detection method to effectively feedback English learners’ fluency. Our method is comprised of sentence stress detection and prediction method. The vowel non-reduction can be found by comparing the two results: predicted and detected stress positions.
Pronunciation variant simulation has similar purpose to the grammar error simulation. Both simulations are required to ensure high speech recognition quality and error detection rate for non-native speaker's speech. Modeling pronunciation error does not require knowledge as much as grammar error simulation. We are developing an error driven learning method to effectively generate English learners' pronunciation variants. Our method is an extension of transformation based learning that is template-free and allows multiple labels for a token.
