In this lesson, you will create a Scratch project that uses the TM2Scratch extension to recognize rock, paper, and scissors hand gestures using a Google Teachable Machine Image model.
Note: You will need a webcam or a camera on your computer to do this project.
In this lesson, you will create a Scratch project that uses a Google Teachable Machine Image model to recognize rock, paper, and scissors hand gestures.
If you have not created your own image model in the previous lesson, you can use the following link for an image model that has been trained to recognise rock, paper and scissors hand gestures.
https://teachablemachine.withgoogle.com/models/wUzRtfVIZ/
First, go to stretch3.github.io to create a new Scratch project.
Stretch3 is an experimental version of Scratch 3 that allows you to use additional extensions like TM2Scratch, which we will need for using our AI image model.
Now, let's add the TM2Scratch extension to your project.
Click on the 'Extensions' button at the bottom-left corner of the screen, and then click on 'TM2Scratch'. This extension allows you to use Google Teachable Machine Image models in your Scratch projects.
When you add this extension, it will automatically try and use your computer's camera. If necessary, click on 'Allow' to give it permission to use your camera. You should see what the camera is showing in the stage area.
Scratch Extensions are additional sets of blocks that expand the capabilities of your Scratch projects. They allow you to:
When you add an extension to your project:
Now, let's set up the image model that our project will use. Add the following code to the Cat sprite:
when green flag clicked
turn video [on v] :: extension
Label once every (1 v) seconds :: extension
image classification model URL [link to your model] :: extension
Put the link of your image model into the image classification model URL
block.
Here's what the code does:
turn video [on v]
: This block turns on the video from your webcam or computer camera.Label once every (1 v) seconds
: This block tells the program to recognize the hand gesture every second.image classification model URL [link to your model]
: This block sets the URL of the image model that will be used to recognize the hand gestures.Click on the 'image label' block in the TM2Scratch extension category to turn it on. This will show you the current image label being applied to what's being recorded on your webcam.