How to Make Your LEGO SPIKE Robot Understand Speech!

Ever wanted to tell your LEGO SPIKE robot to move forward, and it actually does? Welcome to the exciting world of voice-controlled robots! We’re diving into how to make your LEGO SPIKE Prime robot understand speech. For this, we’re using the SEN0539 Voice Recognition Sensor, an LMS-ESP32 board, and Pybricks running on the SPIKE Prime hub.

Here is a video of the final result. Turn the sound on to see how fast the sensor and the robot react to voice commands. This is much faster than Alexa or Google Assistant because you don’t need an internet connection. It’s an offline voice recognition sensor!

What You’ll Need to make your SPIKE Prime robot understand speech

• LEGO SPIKE Prime set or Robot Inventor Set
• SEN0539 Voice Recognition Sensor
• LMS-ESP32 board
• Pybricks installed on your SPIKE Prime hub
• Code from the PUPRemote GitHub repository

Connect SEN0539 voice recognition sensor to LMS-ESP32 board

Start by connecting your SEN0539 Voice Recognition Sensor to the LMS-ESP32 board via I2C. Here’s a quick guide:

1. Connect SEN0539’s green wire to LMS-ESP32’s IO26 pin.
2. Connect SEN0539’s blue wire to LMS-ESP32’s IO2 pin.
3. Connect SEN0539’s black wire to LMS-ESP32’s GND pin.
4. Connect SEN0539’s red wire to LMS-ESP32’s 3v3 pin.
5. Ensure the switches on the SEN0539 are set to SPK1 and I2C.

Your SEN0539 is now ready to pick up voice commands and translate them to data. The LMS-ESP32 board will read this data via I2C.

Program LMS-ESP32 to Emulate LEGO Sensor Protocol

To send this voice data to the SPIKE Prime hub, the LMS-ESP32 needs to “pretend” it’s a LEGO sensor. This involves emulating the LEGO Sensor Protocol.

1. Download the code from the GitHub link above.
2. Upload main.py and sen0539.py to your LMS-ESP32 board with Thonny or your favorite ESP programming software.
3. The code will read voice data from SEN0539 and package it to mimic the LEGO Sensor Protocol.

Alternatively, you can clone the repository with VS Code and use Pymakr to run it on the LMS-ESP32. That’s how I developed the project. All configuration for this is in the GitHub repository.

Get SPIKE Prime Ready with Pybricks

Your SPIKE Prime hub needs to run Pybricks for this magic to happen.

1. Install Pybricks on your SPIKE Prime hub if you haven’t already.
2. Create a new program called pupremote.py
3. Paste the contents of pupremote.py in there.
4. Create another new program with a name of your choosing.
5. Copy the contents of pybricks_drive_voice.py in there.
6. Connect to your hub over Bluetooth and press run!

Here’s a sample code snippet in Pybricks:

# Build a simple robot with wheels on ports C and D
# Connect a voice recognition sensor dfrobot SEN0539 to LMS-ESP32 and then to your hub
# Put the sensor in i2c mode.
# https://wiki.dfrobot.com/SKU_SEN0539-EN_Gravity_Voice_Recognition_Module_I2C_UART

from pybricks.pupdevices import Motor, ColorSensor, UltrasonicSensor, ForceSensor
from pybricks.parameters import Button, Color, Direction, Port, Side, Stop
from pybricks.robotics import DriveBase
from pybricks.tools import wait, StopWatch
from pupremote import PUPRemoteHub

p=PUPRemoteHub(Port.A)
lm = Motor(Port.C)
rm = Motor(Port.D, Direction.COUNTERCLOCKWISE)
db=DriveBase(lm, rm, 56, 120)

p.add_command('listen', to_hub_fmt="b")

while 1:
    ans = p.call('listen')
    if ans > 0:
        if ans == 22:
            print('Fwd')
            db.straight(50)
        elif ans == 23:
            print('Back')
            db.straight(-50)
        elif ans == 25:
            print('Left 90')
            db.turn(-90)
        elif ans == 26:
            print('Left 45')
            db.turn(-45)
        elif ans == 27:
            print('Left 30')
            db.turn(-30)
        elif ans == 28:
            print('Right 90')
            db.turn(90)
        elif ans == 29:
            print('Right 45')
            db.turn(45)
        elif ans == 30:
            print('Right 30')
            db.turn(30)
        elif ans == 2:
            print('OK')
        else:
            print( ans )
        # Wait until something new is said, avoid doubles.
        wait(500)

Now Speak to Your Robot!

Fire up all components, and you’re all set! Speak the commands you programmed into the SEN0539 sensor, and your SPIKE Prime robot will follow.

There you have it! You’ve just turned your LEGO SPIKE Prime into a voice-controlled robot. Imagine the fun and interactive projects you can build now. High five!

Wait a minute… PUPRemote?

You may have noticed the PUPRemote import in the program. It is a new library we designed to make it easy to talk to custom hardware. I will blog about it more. For now: check the PUPRemote documentation!