[ISSUE] mBot Neo voice recognition block is not working

#1

Hey folks. We just purchased an mbot and are successfully running and deploying programs onto the bot with makeblock 5.4.3. As a software engineer, I was pretty eager to teach my daughter how to get this thing to respond to her voice, so I was excited to see the AI set of blocks, including voice recognition and text to speech. This seems perfect… except that I can’t get the recognize block to do anything.

I’ve followed a sample to no avail. My program is basically this:

  • Starting event: when button A is pressed
  • connect to WiFi
  • wait until network is connected
  • recognize [english] [3] seconds
  • show label [1] [speech recognition result] at [center of screen] by [big] pixel

In theory, this is very simple… it should connect to the wifi, attempt to recognize english for 3 seconds, then show what it recognized on the display of the CyberPi. Everything works except for the recognize block. I can’t figure out what’s going wrong because I can’t find any documentation for the actual recognize block.

If I remove the recognize block and just add some static text instead of the recognition result, like “Hello World!” it works fine obviously. I’ve been working with the Upload mode since the AI blocks only seem to work with Upload mode.

Has anyone recently had success with the recognize mode?

#2

I have no issues. I recommend a code like this:

image
image.png413x702 48.4 KB

So that you can better tell what is going on. Connecting to Wi-Fi will take a minute, and the voice processing API can be rather slow as well. I hope this helps you, @rob_todd!

Best_codes

#3

Here is my project file:

mBot Speech to text.zip (200.6 KB)

It’s a ZIP file right now, so download it and simply rename the “zip” part to “mblock” and open it with mBlock. Good luck!

#4

The code isn’t the problem. The block isn’t working. I’m having the same issue as the OP even using the code provided above.

I also tried 1-5 second waits.

Also tested sound recognition and that works. So, it can hear us.

#5

It’s working fine for me. If you have more issues, you may need to downgrade to mBlock 5.3.0 or 5.4.0 instead of 5.4.3. @sluffmo @rob_todd

#6

Yeah I’ve done basically exactly this program, it simply doesn’t work. I’ll try downgrading to see if that helps at all.

#7

Awesome, let me know how it goes! :slight_smile:

#8

no luck. I also tried the web version, also no luck.

#9

Hmm. I’ll contact makeblock about the issue. You can too at support@makeblock.com. Sorry for the trouble, and happy coding!

@rob_todd

#10

Your program just prints “Listening…” over and over between screen clears

#11

If you start it, wait till connected, Push A, wait for “listening”, and then wait about a minute for the result? (Just in case).

#12

Nope, no luck. I’m thinking that there’s some kernel incompatibility with this version of my mbot, like it can’t find the microphone device or something.

#13

Maybe. I guess we can only just troubleshoot a bit ourselves and contact mBlock. @rob_todd Good luck!

#14

@rob_todd - Did you receive a fix or resolution for this issue from customer service? I am having a similar issue and have not been able to resolve it either. I contacted Customer Service via the email address.

#15

@matt_ailabjunior I have no issues, but I’ll see what I can find out. Can you provide some more info to help troubleshoot? Thanks!

#16

@everyone What is the firmware version of mBot 2 and what version of mBlock are you using?

#17

Same issue as rob_todd. The voice recognition service does not seem to be working “out-of-the-box”. I have confirmed the mBot Neo is connected to the Internet by testing other services such weather and temperature for the IoT services. I can confirm it is sending data to the Internet when I send a request for the Voice Recognition, but have not set up a Proxy to intercept and decode the API request yet (which is my next step).

I have used the mBlock Web and desktop Apps (even back installed prior versions to 2001 based on their releases).

What version of mBot2 firmware are you using? I think the mBlock App upgraded mine to 44.01.011.

#18

@matt_ailabjunior I’m using the same firmware version as you. It looks like this might be an issue with makeblock’s API, so I’ll have to contact them about it. Thanks for helping troubleshoot!

In the meantime, we can look for other potential issues or a workaround. Good luck!

#19

Is there a way to know where the API endpoints to test with the API Key (Cloud Service Auth Code) in MakeBlock. I have been reviewing the Python API documentation on Archive.org as I found references to them on Google, but no longer active on their website.

https://web.archive.org/web/20211027064138/https://education.makeblock.com/help/category/mblock-python/mblock-python-editor-python-api-documentation-for-devices/mblock-python-editor-python-api-documentation-for-cyberpi/

#20

@matt_ailabjunior I’ve looked quite a bit too, but I think they made their API stuff intentionally a bit hard to find. You could very technically set up a special Wi-Fi router to connect your mBot to and have it intercept and log all traffic through it. The APIs are usually something like *.mblock.cc, like this one:

https://extservice.mblock.cc/

I’ll look around when my mBot gets charged, I’m charging it right now. :slight_smile: Good luck!

I have a question for everyone @matt_ailabjunior @rob_todd @everyone are you signed in to with your account from mBlock.cc (passport.mblock.cc) when uploading these codes?
image
Just making sure.

About all that I could find in the docs for this was:

AI

CyberPi provides some AI capabilities. After being connected to the Internet, CyberPi can recognize speeches, read texts out loud, and translate some texts through the corresponding AI interfaces.

cyberpi.cloud.listen(language, t=3)

Starts to recognize a speech in the specified language

This API blocks the thread until the speech is recognized.

Parameters:

language: str, language of the speech to be recognized
Setting range:


"chinese": Chinese

"chinese_taiwan": Taiwan Mandarin

"cantonese": Cantonese

"japanese": Japanese

"english": English

"french": French

"german": German

"spanish": Spanish

"portuguese": Portuguese

"russian": Russian

"korean": Korean

"italian": Italian

"dutch": Dutch

t: float or str; the default value is 3
When you set t to int, it indicates the period for recognizing a speech, in seconds; when you set it to str, you can set it only to record, indicating that the last recorded audio file is used for speech recognition.

cyberpi.cloud.listen_result()

Obtains the last recognized speech

A string value is returned.