National University of Ireland, Maynooth Maynooth, Co. Kildare, Ireland. Department of Computer Science Technical Report Series NUIM-CS-TR

National University of Ireland, Maynooth Maynooth, Co. Kildare, Ireland. Department of Computer Science Technical Report Series NUIM-CS-TR2002-02 The Computer s Role in Speech Therapy Jackie O Kelly http://www.cs.may.ie/ Tel: +353 1 7083847 Fax: +353 1 7083848

Jackie O'Kelly Department of Computer Science National University of Ireland, Maynooth Date: February 2002 Technical Report: Key words: Speech and Language Therapy, Speech Disorders, Articulation, Phonemes, Visual Speech. Abstract Computer-based speech therapy aids can be seen as clinical tools. Speech therapy software can aid diagnosis of speech disorders, and provide visual feed back during treatment. A visual display can be useful for encouraging patients e.g. gives them target / goals to aim for (in terms of improving their speech), it can enable a deaf patient to see their progress. There are a number of software tools available. Some are simple and provide a single type of speech display, while others are more complex, with a variety of display formats that can be used for training multiple aspects of speech. The software can be categorised into software that deals with speech and software that deals with literacy.

This report gives a summary of software that is currently available for use in Speech Therapy. It gives a brief overview of speech and the role of the speech therapist. It also identifies the types of speech problems that the therapist works with. It then gives a brief summary of each of the products that were evaluated. In most instances this evaluation was conducted on demonstration versions of the products and these offer restricted access to features of the product. In other cases no demonstration software was available and therefore some information in the report is related to what is available on the web site for the product or in the marketing literature for the product. The software can be broken down into two distinct areas: (See Table 2 and 3) ƒ ƒ Software that deals with speech. Software that deals with issues of literacy. All of the products looked at will run on a standard PC with a sound card, graphics card, speakers and microphone. Given that systems today come with these features as standard means that the average PC is all that is required with the software. The software evaluated ranges in price (see Table 1) from FREE to $2,495 and in some cases different prices for the client (a lower price, with less features available) and the therapist (a higher price with more features available). The software can be looked at as a tool or aid for the speech therapist and not a replacement for the therapist. Table 1 lists the software products, supplier and costs. Table 2 lists the software that deals with speech and identifies some of its key features. Table 3 lists the software that deals with literacy and identifies some its key features. 2

What is Speech? Speech is a product of the interaction of respiratory, laryngeal and vocal tract structures. The larynx functions as a valve that connects the respiratory system to the airway passages of the throat, mouth and nose. The vocal tract system consists of the various passages from the glottis to the lips. It involves the pharynx, oral and nasal cavities, including the tongue, teeth, velum and lips. The product of speech sounds through these organs is known as articulation. Speech is the spoken medium of language and one of the primary ways we communicate with our environment. It is also an effective way to monitor normal growth and development as well as identify potential problems. Phonology is a branch of linguistics that is concerned with the study of the sound systems of languages. The aims of phonology are to demonstrate the patterns of distinctive sound contrasts in language, and to explain the ways speech sounds are organised and represented in the mind. Phonological development is the gradual process of acquiring adult speech patterns. Articulation is a general term used in phonetics to denote the physiological movements involved in modifying the airflow, in the vocal tract above the larynx, to produce the various speech sounds. Sounds are classified according to their place and manner of articulation in the vocal mechanism. The use of speech to communicate is unique to humans. When speech is impaired or absent, the impact on the person is profound. What is Speech and Language Therapy? Speech and Language Therapy is the health care profession specifically concerned with the identification of communication disorders and the provision of therapy for people with communication impairments. Communication impairments include:- Disorders of Articulation which affects the intelligibility and effectiveness of speech in varying degrees. Disorders of Language in which use and understanding of the spoken word may be impaired. Disorders of Fluency including stammering in which the normal continuity of speech is interrupted with varying degrees of frequency and severity. The main goal of speech therapy is to restore the ability to communicate accurately. 3

Speech Disorders Speech disorders are a group of disorders that result in impaired or ineffective communication. Dysfluencies are rhythm disorders that are usually characterised by the repetition of a sound, word, or phrase. Stuttering is, perhaps, the most serious dysfluency. Articulation deficiencies are sounds that are made incorrectly or inappropriately. Voice disorders involve differences in the quality, pitch, and loudness of the sound. The Role of Computers Computer-based speech therapy software can be seen as clinical tools. The speech therapist uses his/her clinical skill in conjunction with the computer, to advise and encourage the patient. Games can be used to help speech therapy, especially in children. Patients who are undergoing radical surgery can record their own voice prior to the surgery, and subsequently this can be used as a goal for their therapy after the surgery is complete. Computer-based speech therapy software can:- Aid diagnosis of speech disorders Provide visual feedback during treatment. Visual display is very useful for encouraging patients e.g Gives them target/goal to aim for (in terms of improving their speech). Deafness in patients - enables them to see their progress. Reduce costs of seeing a speech therapist. Monitor and assess speech progress Provide practice exercises for unsupervised patients/clients. There are a number of software tools available. Some are simple and provide a single type of speech display, while others are more complex, with a variety of display formats that can be used for training multiple aspects of speech. The software can be categorised into software that deals with speech and software that deals with literacy. (See Table 2 and Table 3). 4

Introduction to current state of knowledge in Speech Therapy Software Articulation I, II, III Articulation I, each target phoneme has 20 words in each of the initial, medial and final positions. Articulation II and Articulation III allow you to choose your own sets of categories and save them for later use. Deals with Consonant Phonemes (I), Consonant Clusters (II), Vowels + R and R Clusters (III). AphasiaMate AphasiaMate consists of eight main sections containing 146 different therapy tasks and more than 1700 individual exercises. Exercises incorporate text, graphics, and video clips and work on skills including comprehension of written and auditory material, spelling, solving problems of time and money, and classifying semantically-related items. Tasks range from a basic to high level of difficulty, so AphasiaMate may be used repeatedly by clients from early to late stages of treatment. Key Benefits is simple and easy for therapists and clients to use tasks are highly relevant to everyday communication skills clear instructions for each task are provided in both written and audio form detailed client results are automatically recorded and saved when required tracking progress is easy since results for different sessions are all saved together for a given client accuracy feedback is available after completion of each task, and for each individual exercise the program can be used on a single computer by multiple therapists and clients therapist can select exercises prior to the session, and program will automatically run through them is an efficient use of therapy resources AphasiaMate's Main Menu lists the eight areas in which tasks are organised. 5

The Computerised Home Aphasia Therapy (C.H.A.T) The C.H.A.T. program was developed primarily for people with aphasia who have been discharged from a formal language retrieval therapy program but who feel that they could benefit from more therapy if given more time. The C.H.A.T. program consist of 220 lessons covering sounds, words, phrases, sentences, Melodic Intonation Therapy, Minimally Prompted Speech and narrative. Chatback This program allows the teacher to present the pupil with recorded examples of words and listen to how they repeat them, which will be of interest to speech therapists and any teachers working with pupils with auditory discrimination problems. The aim is to let the pupil listen carefully to a word and compare their own attempt to reproduce the sound. Visual display of phonemes A useful feature of the program is a display of a mouth saying the appropriate phonemes. When required, this will 'mouth' the sequence of sounds making up a word. The editor supplied with Chatback enables the teacher to quickly prepare new word lists. 6

Dr. Speech (Speech Therapy) Speech Therapy uses over 30 voice-activated video games to provide real-time reinforcement of a client's attempts to produce changes in pitch, loudness, voiced and unvoiced phonation, voicing onset, maximum phonation time, sound and vowel tracking. Speech Therapy is divided into two groups: 1) Awareness teaches children about the attributes of their voice, and 2) Skill Builder gives the user goals to achieve for a given range and time. Comprehensive User Logs calculate therapy time and track the client's progress. Real-time recording and playback give you the tools needed to maximise the client's therapy. Earobics Systematically teaches the critical phonological awareness, auditory processing and introductory phonics skills required for learning to read and spell. The games also develop general cognitive skills that support learning, such as attention and memory, sequential memory, auditory performance with competing signals, phoneme discrimination and identification, phonological sequencing, sound-symbol correspondence. Ecos/Win Getting Started -- Signal Recording, Editing & Saving Spectrography -- the Basics Wide-band and Narrow-band Spectral Analysis Visual Representations of Speech: Spectrograms Spectral Analysis: Spectrograms, FFTs and LPCs Vowels and Diphthongs Fricative/Affricate Contrasts Fricatives Perceptual Context Effects: The SUSHI Experiment Sonorants -- Nasals and Approximants Plosives and Affricates Linear Filtering of Speech Categorical Perception of Voice Onset Time Temporal Factors in Speech Perception Hearing Aid Processed Speech Effects of Hearing Loss on the Intelligibilty of Speech Signals 7

FastForWord Language Is an interactive, computer-based training program that builds the fundamental language skills critical for reading success and communicating in and out of the classroom. FastForWord Language stimulates rapid language skill development as children learn to distinguish the various components of speech. The program consists of five 20-minute training sessions a day, five days a week. Results from the exercises are analysed daily and compared with the student's progress to date. Letter Sounds Letter Sounds is a free program that provides young children an introduction to the idea of letter sounds and phonics. Children drag a picture to the letter that the word that describes the picture starts with. Listen-Hear A system to help children develop their spoken language skills, the program facilitates auditory perception and language learning following pediatric cochlear implantation or hearing-aid fitting. The program, uses a game format targeted toward children aged three to ten. The common aspect of all games is the requirement to use auditory signals to do the task. Children who have a hearing loss are at significant risk for delays in language development, reading and educational achievement. Reducing the impact of hearing loss depends on early detection and effective intervention. The Listen-Hear multimedia package uses the procedures developed by leading AV therapists. 8

Literarcy Box (Year 1 & 2) Year 1 Key Content: Recognising rhyming patterns Identifying initial and final phonemes in CVC words Blending phonemes to read CVC words Spelling words containing initial consonant clusters Identifying, reading and spelling long vowel phonemes Year 2 Key Content: Reading and writing the common spelling patterns for vowel phonemes Identifying, reading and spelling long vowel phonemes Spelling compound words Identifying syllables Spelling CVC words and words containing consonant clusters Essential Literarcy Learn the basic rules for reading and spelling by identifying the vowel sound in a word, sounding out the word, and identifying the rule. Develop stronger phonemic awareness and auditory processing skills. Phonic Word Builder This software gives assistance to the teaching of phonic skills, reading, spelling and support for the Literacy Hour. Divided into sections, with bright and colourful interactive screens, the child is motivated to construct words using the wide vocabulary included in teaching phonics such as 'ed', 'st', 'ing' and magic 'e', as well as building simple sentences. Supplied with over 150 phonic activities, a key feature of the software is that it allows the teacher to create new and edit the existing phonic activity files. This makes it ideal for tailoring to the schools'/pupils' individual needs. ProTrain The ProTrain system is used to improve speech production skills by allowing talkers to see and hear key differences between their speech and target (correct) speech samples. Designed for the treatment of phonology and articulation disorders, voice disorders, and foreign accent reduction, ProTrain displays specified parameters of speech to assist clients in reducing speech errors. 9

React React is a program of interactive tasks designed to augment the speech and language therapist's existing repertoire of dysphasia therapy materials. One of its features is the use of video to work on understanding conversations. A log-on screen records the client's name, his/her unique identification number and (optional) the name of the therapist. The user can move into and between tasks with clear written and audio instructions at the start of each task. The program is simple and intuitive to use via a menu system. Click on a section and then chose a task from the menu. An on-screen help system provides assistance throughout the program. The therapist can either use the standard menu system or pre-select individual tasks enabling the session to be tailored to meet the specific needs of the client. As the user works through the program an on-screen coloured bar shows the progress through a task. Within each exercise the user will automatically be shown the correct answer after two failed attempts. At the end of each task the user can opt to see an instant visual display of his/her percentage score. In addition, at the end of a session, more detailed results viz. the score and time taken to complete each task can be viewed on-screen, by selecting the client's unique identification from a "drop down" list. Results of individual sessions can be viewed or printed for audit purposes. Sails Speech Assessment and Interactive Learning System Effective treatment for children's articulation and phonological disorders, within highly motivating and enjoyable computer games. Phonemic Training The foundation of phonological and articulation disorders is often the child's inability to hear and use the difference between correctly and incorrectly produced target sounds. SAILS treatment targets the perceptual difficulties that contribute to most common childhood articulation and phonological errors. SAILS also quickly yet accurately assesses whether the child is likely to benefit from treatment. Effective Treatment Tool SAILS has been proven to be effective through more than a decade of clinical experience and in clinical research trials. Treatment with SAILS is commonly followed by rapid and substantial improvements. SAILS treatment is often combined with production training (e.g. using ProTrain, see separate sheet). Incorporates Natural Speech SAILS uses adult and child speech samples from different talkers to promote generalisation, and to ensure that an appropriate range of naturally-occurring tokens of the problem sound are presented accurately. SAILS incorporates 16 modules representing the most common childhood speech errors. Additional modules are easily developed using our ModGen (Module Generator) software, which is included with the SAILS program. On-Line Record Keeping The program keeps track of all results, allowing for easy progress tracking. Results may be incorporated into clinical reports using word processing software. 10

Sight n'sounds Speech therapy software that helps to improve articulation and word retrieval. It is useful for aphasia and oral apraxia. Displays a word and/or picture and speaks the word (using prerecorded human speech). Then records the user speaking the model and plays back the users' speech and the model speech for comparison. The professional version allows you to enter your own words. There are different levels of use: 1: Single syllable words, 2: Short words, organised by beginning sound, 3: Short words organised by ending sound, 4: Pictures and words for verbs, 5: Words for abstract Sounds and Rhymes Starting with single sounds and culminating in the identification of rhyming words, a progression of tasks aims to develop recognition of short vowels and automatic response to sounds within words. The Sounds task concentrates on the standard five vowel sounds, presenting several practice formats varied between the pupil listening to the initial sound of a word and matching it to a vowel or grouping simple three letter words by their middle letter. The Rhymes task covers many topics - initial and final consonant, word endings, rhyming words, matching words, odd word out and others. Each of the tasks can be practised with or without the aid of textual clues, making the task purely auditory if required. Speech Sounds on Cue Provides multimedia full-motion video cues, for independent speech practice. Cues are for individual phonemes (sounds) as well as full words. This program is especially helpful to those who need to hear and see a sound made. Displays cues for over 500 words, and 20 consonant positions: Colour photo Auditory and written trigger phrase Written word Video clip of a speaker User advances through each exercise in a lesson with the mouse or keyboard. He/she can also repeat an exercise as many times as they want. Also has a print option for colour photographs of 20 consonant articulatory positions. All words and their trigger phrases can also be printed in a booklet. Speech Prism SpeechPrism addresses all aspects of learning speech production from learning simple isolated vowel and consonant sounds, to using those sounds in the context of words and phrases. SpeechPrism provides many different ways of viewing speech sounds in real time. Two simultaneous views are provided so that one can be used as a prototype or template for learning. Applications of this software range from teaching hearing-impaired speakers to treating communications disorders to accent reduction. 11

Speech Viewer III SpeechViewer III is a speech and language tool that transforms spoken words and sounds into imaginative graphics. SpeechViewer III increases the effectiveness of speech therapy and speech modification for people who have speech, language or hearing impairments. You can select from over a dozen SpeechViewer exercises. Each exercise responds to your voice input with immediate, clear and meaningful feedback that helps you see how to speak. In addition, SpeechViewer III provides animated rewards that reinforce successful responses. The software incorporates SVGA graphics, a revised menu bar format for access to clinical exercises, point-and-click capability, pull-down menus, and enhanced phoneme (a word, phrase, or sound) model creation. The program's clinical management functions automatically collect pertinent data that allow therapists to easily document progress and see and hear results. SpeechViewer III is designed to help people of all ages who have a variety of disabilities, such as speech or language impairments, cerebral palsy, developmental delay, traumatic brain injury, and speech disorders resulting from a stroke. It is ideal for professional speech language pathologists, special education teachers, teachers of the deaf, English as a second language instructors, professionals working with accent reduction and others. Features Fun tutorial by a speech/language pathologist Extensive online help Menu grouping of exercises Easy to use Point-and-click access Drop-down menus Powerful attention-grabbing exercises Captivating, creative animations Graphics for all age ranges Visual feedback of speech production (Acoustics) including, loudness, pitch, timing patterns, voicing, sustained phoneme production, waveforms, spectra, and spectrograms Powerful note taking and reporting Create notes like you would on a sticky pad Add audio clips Add statistics Pick and choose notes to print or file Powerful phonological tools Fine tune models Save best productions Create models from saved speech samples 12

Speed Pacer Speech Pacer moves a text-highlighting cursor along to pace a user s reading rate. Key Features and Benefits Automated therapy...automated cursor advance Clinician can set the pace that the user reads and then let the user work on their own. Unlimited therapy... Pro version will read any normal text file for word and sentence pacing. You can use the reading files included or use your own reading files. SpeechPacer will work with any normal text file up to 10K. (Larger files are shortened). The phrase pacing requires that the file be marked with the breaks between phrases. The ReadingCursor paces by word, phrase or sentence. The font size can be changed for users with vision problems. VideoVoice Video Voice offers facilities for almost any type of speech work, with more than 20 games and colourful graphic displays with 50 variations that give you new ways to illustrate speech. Represent pitch as colours in a spectrum... give "speeding tickets" for too much volume... teach sounds with a "gobble-the-dots" game... create pictures that show co-articulation and stress in connected speech. Video Voice is appropriate for many age and ability levels because you control the content and tailor operation to best meet individual needs. One person might use it for simple vowel production, and another more complex aspects of connected speech. What sounds? Which words? That's up to you! Built-in record-keeping and reporting capabilities help quantify performance and track progress over time. 13

Vowel Target Vowel Target is a highly interactive game for learning American English vowel articulation (including the semi-vowel "er" as in bird). Specially designed to help language-impaired children. This game allows the player to adjust his or her tongue position in real time based on the graphical feedback provided by the game. Two kinds of graphical feedback are provided; the position of arrows that appear as the student speaks, and an animated head cross section that shows an estimate of tongue position. The game uses an engaging archery theme to hold the student's interest. Scoring and sounds provide incentives for performance. Vowel Target has also been successfully used for foreign accent reduction, by placing the vowel sounds in the context of American English words during the game. 14

Table 1 Software Product Listing Title Supplier Cost Articulation I,II,III www.locutour.com $99.00* AphasiaMate http://www.avaaz.com $1,049 C.H.A.T. http://www.aphasia-therapy.com $169.50 ChatBack http://xavier.bangor.ac.uk 35 (stg)** Dr. Speech (Speech Therapy) Charlemagne s.a.r.l. Not listed Les hauts de Magny 4 22 chemin de magny 0128 Prevessin, France Earobics http://www.earobics.com Specialist & Clinicians $299 per product Home Users $59 per product EcoS/Win http://www.avaaz.com $599 FastForWord Language www.scientificlearning.com $269 Letter Sounds www.yourchildlearns.com FREE Listen-Hear http://www.avaaz.com 3 modules @ $125 each Literarcy Box (Year 1) (Year 2) http://sales.sherston.co.uk 40 (stg) Essential Literarcy www.locutour.com 189 (stg) Phonic Word Builder www.neptunect.co.uk 25 (stg) for single user 150 (stg) for network site licence ProTrain www.avaaz.com 225 (stg) React www.avaaz.com 425 (stg) Sails www.propeller.net 275 (stg) Sights n' Sounds http://www.bungalowsoftware.com Standard $129.50 DeLuxe $149.59 Professional $189.50 Sounds and Rhymes http://xavier.bangor.ac.uk 36.50 (stg) Speech Sounds on Cue http://www.propeller.net/react 90 (stg) Speech Prism http://www.bungalowsoftware.com $149 (1 PC) $269.50 (3 PCs) SpeechViewer III 695 (stg) Speed Pacer http://www.bungalowsoftware.com $69.50 (1 PC) $119.50 (3 PCs) VideoVoice http://www.videovoice.com $2,495 Vowel Target http://www.bungalowsoftware.com $59.50 (1 PC) $99.50 (3 PCs) * American dollars ** Pounds sterling All sites where live at the time the report was compiled. 15

Table 2 Speech Software Listing Title Phonology Formant Display Pitch Prosody Voicing Reports/ Analysis Clinical Tools Articulation I,II,III X C.H.A.T X X ChatBack X X Dr. Speech X X X X X Earobics X X FastForWord Language X Essential Literacy X Phonic Word Builder X ProTrain X X X Sails X X Sounds and Rhymes X Speech Sounds on Cue X Speech Prism X X X X SpeechViewer III X X X X X X X VideoVoice X X X X X X X Table 3 Literacy Software Listing Title Auditory Proc. Sentence Proc. Time Spelling Reading Semantic Visual Matching AphasiaMate X X X X X X X Earobics X X X EcoS/Win FastForWord Language X X X X Letter Sounds X X Listen-Hear X X Literacy Box (Yr 1) (Yr2) X X X Essential Literacy X X X X Phonic Word Builder X X ProTrain React X X X X X X X Speech Pacer X X X 16

References Royal College of Speech and Language Therapists http://www.rcslt.org Irish Association of Speech and Language Therapists http://www.clubi.ie/iaslt American Speech-Language Hearing Association http://www.asha.org Learning Fundamentals http://www.locutour.com Aphasia Therapy Products http://www.aphasia-therapy.com Xavier Educational Software Limited http://xavier.bangor.ac.uk Cognitive Concepts http://www.earobics.com AVAAZ Inovations http://www.avaaz.com Scientific Learning http://www.scientificlearning.com Owl & Mouse Educational Software http://www.yourchildlearns.com Sherston Software Limited http://sales.sherston.co.uk Neptune Computer Technology http://www.neptunect.co.uk Propeller Multimedia Limited http://propeller.net Bungalow Software http://www.bungalowsoftware.com Video Voice http://www.videovoice.com Acknowledgements The author would like to thank Dr. Pauline Sloan, Senior Lecturer in Speech & Language Pathology, Trinity College Dublin and Ms. Clothra Ní Cholmáin, Speech Therapist, Speech & Language Pathology, Trinity College Dublin for their views on the applicability of computer based software in speech and language therapy. 17