American Sign Language Alphabet App (Project #2) (Version 2 of App Inventor) Description:

Transcription

1 American Sign Language Alphabet App (Project #2) (Version 2 of App Inventor) Description: This App will take text input and then translate each letter of the text to the corresponding American Sign Language Image. We will also use the TextToSpeech object in App Inventor to speak each letter as it is translated by the App. Part 1: User Interface Design

2 Process: 1. Go to and log in with your Google Account if necessary. 2. Select My Projects -> Start new project 3. Name the project ASLAlphabet. Click OK.

3 4. You will be presented with a blank App Inventor Project: 5. Now it is time to design the User interface. First, Click on User Interface on the Palette and drag an Image object to the screen.

4 6. Note that the Image1 object is small and in the upper left corner. This is acceptable for now. The Image object will grow once we attach an image resource. We will change the name of the object. Click on the Image1 in Components and select Rename. Change the name to imageview and click OK.

5 7. Now drag a Label object from the Palette. 8. We need to adjust the properties of the Label1. First, click on Label1 on the Components and set the Width property to fill_parent. Click OK.

6 9. Change the Label1 Text property to no text. (Delete Text for Label1 in the Text field in Properties).

7 10. Change the name of Label1 to textphrase by using the Rename button.

8 11. Next, drag a TextBox object to the Phone screen area: 12. Set the following properties for the TextBox1: a. Set the width to fill_parent b. Set the Hint to Enter Phrase c. Rename to editphrase

9 13. We will now add a Button that the User will press to start the translation. Drag a Button to the phone screen.

10 14. Change the following properties on the Button: a. Change the width to fill_parent b. Change the text to Translate c. Rename to buttontranslate

11 15. We will now add a button for the User to press and have the phone speak the phrase. Drag another button to the screen area.

12 16. Change the properties of the Button to: a. Change the width to fill_parent b. Change the text to Speak c. Rename to buttonspeak

13 17. We need two non-visible objects for the App functions. Drag a Clock object to the phone screen area and rename it to timer. Note that the timer shows up as a non-visible component. Uncheck the TimerEnabled Property. Uncheck this box!

14 18. Click on Media on the Palette and drag a TextToSpeech object to the phone screen. Rename the object to texttospeech. Again, note that it is a non-visible component. 19. We are finished with the User Interface design. In the next section we will upload the ASL Alphabet images used in the Application.

15 Part 2: Image Resource Upload Image and sounds play a vital role in Assistive Technology Applications. For students who cannot read, the use of images and icons empowers the user to use the Application. In App Inventor, images and sounds are stored in the Media section. We will upload gif images of the 26 letters and space for the American Sign Language Alphabet. Process: 1. Download the following.zip file and extract: (For this example, I will extract to the downloads folder) 2. Go back to your ASL App project and find the Media section. Click Upload File 3. Select Choose File

16 4. Navigate to where you saved the aslalphabet images folder and select the a.gif image. Click Open 5. Click OK

17 6. Note that now the a.gif file is located in the Media resources:

18 7. We will now map the a.gif to the imageview object. Click on imageview on the Components section. Then click on Picture in the Properties and select a.gif and click OK.

19 8. Now upload the remaining alphabet pictures to the Media area. ( b.gif, c.gif...) (Yes, you have to do this one image at a time..sorry...). 9. You are done with the image uploads. We will now begin to program the App!

20 Part 3: Programming the ASLAlphabet App. Description: The algorithm for this project depends on the concept of the App moving letter by letter from the input phrase and then showing the corresponding picture:

21

22 In order to do this, we will use the Computer Science idea of an Array to hold both a list of the picture names and individual letters: Array: aslimages Array: aslletters Note that the position of the image and the letter correspond (a is 1, b is 2, c is 3...)

23 Process: 1. Click on the Blocks button to go to the block programmer.

24 2. We need to create some variables to help control the flow of the App. Click on Variables and drag an initialize global block to the programming area. 3. Change the name of the variable to letterindex

25 4. Drag a number value 1 to the socket in letterindex. This is like saying letterindex = 1. (You find the number blocks in Math ) 5. We now need to make a Text variable that will store the text as we translate the world. Drag a new initialize global block into the programming area and set the name of the variable to displaytext. 6. Drag a Text block and connect it to the socket in displaytext. (We want the text to be empty this the empty quotation marks).

26 7. Let s run a quick experiment. We want the App to Speak when the buttonspeak is clicked. This is called an Event in programming (when the user clicks a button.. ). Click on buttonspeak on the Blocks and drag a when buttonspeak Click block to the programming area.

27 8. Now click on the texttospeech object and drag a call texttospeech.speak block and place it inside the buttonspeak Click block.

28 9. The texttospeech.speak block has a socket for a message. We want to speak the text the User types into the editphrase box. So, click on editphrase and drag an editphrase.text block to the message socket.

29 10. We are ready to test part of the App! See for detailed instructions on connecting to an emulator or a phone. For this tutorial, I will describe how to connect to an emulator. Make sure you have aistarter running on your computer. On Windows you will see a screen that looks like: 11. Click on Connect and Select Emulator

30 12. You will see a message like this: Be patient, it will take the Emulator a few minutes to start on the first try. 13. You will now see messages informing you that the emulator (or phone) is loading AiCompanion.

31 14. When the Emulator (or device) loads the App it should look like:

32 15. Type a phrase into the Enter Phrase box and click the Speak button. The App should talk! Hello Mr. Michaud! 16. The fun ensues... (Students love to make their phones talk) Now we will continue with the translation part of the project.

33 17. We now define and build the Array that holds the references to the images. Drag a new initialize global block to the program area and change the name to aslimages. 18. Click on Lists on the Blocks and connect a make a list block to the socket of the aslimages block. 19. Place a text block in the first socket of the make a list block. 20. Type a.gif in the text block. 21. Place an additional text block with b.gif in the next socket:

34 22. We are now out of sockets. We need to add additional sockets by clicking on the blue squares in the make a list block. 23. Drag the item block into the list bracket in the dialog. (You will see three item blocks after the drag).

35 24. Notice that after you drag the item block an additional socket appears on the make a list block. Drag another item to create another socket. 25. Place a c.gif and a d.gif in the make a list block. 27. Now create a new list called aslletters (drag an initialize global block and a make a list block.)

36 28. Repeat the previous procedures and add a, b, c, and d text blocks to the aslletters array. 29. We are going to leave these Arrays at a through d for now. Eventually, all the letters have to defined in each Array. The directions will continue with the script to identify and display the ASL images.

37 30. Click on the timer object and drag out a when timer.timer block: 31. The timer.timer script contains the algorithm that will: -identify the current letter in the word -add the current letter to the displaytext variable -display the displaytext in the textphrase object -Find the matching picture from the aslpictures array. -Display the matching picture -increase the letterindex count -check to see if the count is greater than the phrase.

38 32. To start, create a new variable named currentletter and set it to blank text: 33. In the timer.timer block, drag a set currentletter to block inside the timer.timer block. 34. In the Text blocks, drag a segment block into the socket of the set currentletter block:

39 35. Place an editphrase.text block in the text socket of the segment block. 36. Place the get letterindex variable value block into the start socket. 37. Place the number 1 in the length socket of the segment block. 38. Place a set global displaytext block in the timer.timer block.

40 39. Go to Text and place a join block in the socket of the set global displaytext block. 40. Connect a get global displaytext block to the first socket of the join block. 41. Connect a get global currentletter in the second socket of the join block.

41 42. We now want to speak the current letter. Place a texttospeech.speak block and a get currentletter block in the timer.timer block. 43. We now want to display the ASL image of the currentletter. Drag a set imageview.picture To block into the timer.timer block. (Look in the imageview object).

42 44. Go to Lists and place the select list item block into the imageview.picture block. 45. Place a get global aslimages block into the list socket.

43 46. Drag a index in list block from Lists and place it in the index socket. 47. Place a get global currentletter block in the thing socket.

44 48. Place a get global aslletters in the list socket. 49. Now we need to advance the letterindex to the next letter. Place a set letterindex block in the timer.timer block.

45 50. To add 1, place an addition block in the set global letterindex socket. Place a get letterindex in the fist space and a 1 in the second. 51. Now we need to check to make sure we stop the timer when the letterindex is greater than the length of the phrase. Place an if block in the timer block. ( if blocks are found in the Control category).

46 52. Place a > block in the if socket. 53. Place a get letterindex in the first socket of the > block.

47 54. We are comparing to the length of the editphrase text. Click on Text and place a length block in the second socket of the > block. 55. Now place an editphrase.text block in the length socket.

48 56. We now need to program the then part of the if statement. First, we will set the timer.timerenabled to false to stop the timer. 57. Now set the value of letterindex to 1 so we can start over.

49 58. We need to add a block in the middle of the sequence to display the displaytext in the textphrase object. Add the following blocks (indicated by the arrow).

50 59. Congratulations you finished the timer.timer block! We need to set up one more event with the buttontranslate to finish the App. Drag a when buttontranslate clicked to the program area. 60. Place the following block expressions into buttontranslate.click: a. displaytext = (Blank) b. letterindex = 1 c. set timer.enabled to true 61. Congratulations! You finished the App. Test on the emulator to see if it works!