Smart Camera, number of block detected

#1

Hi,

We try to you use the smart camera using mblock.
We have few questions:

  • How we can know the number of color detected?
  • How can we choose wich signal we whant use when the camera detect more than one?

Regards
Frederic

#2

This is an automated AI response. Verify important information.

@FredRouv
To know the number of colors detected by the Makeblock Smart Camera in mBlock 5, use the “get blocks” block to get the count of detected color objects. To choose a signal when multiple colors are detected, use the “get block {variable}” block to access each block’s details (e.g., signature, position, size) and select based on criteria like size or specific color.

Reasoning and research process

Key Points

  • It seems likely that to know the number of colors detected by the Makeblock Smart Camera in mBlock 5, you can use a block like “get blocks” from the Smart Camera extension, which returns the count of detected color objects.
  • Research suggests that to choose which signal to use when multiple colors are detected, you can access each detected block’s details (e.g., signature, position) using a block like “get block {variable}” and select based on your criteria, such as size or specific color.

How to Know the Number of Colors Detected

The Makeblock Smart Camera, when integrated with mBlock 5, can detect up to 6 learned colors. To find out how many colors are currently detected, add the “Smart Camera” extension in mBlock 5. Look for a block that likely says “get blocks,” which should return the number of color blocks (objects) the camera sees at that moment. For example, if it returns 3, it means three learned colors are detected.

How to Choose Which Signal to Use When Multiple Colors Are Detected

When the camera detects more than one color, you can use blocks from the Smart Camera extension to handle each detected color individually. Use the “get block {variable}” block to get details like the color’s signature (which color it is, e.g., 1 for the first learned color), its position (x, y), and size (width, height). You can then decide which color to focus on based on these details, such as choosing the largest color block or one with a specific signature. For instance, you might check if the block with signature 2 is present and use its position for your robot’s movement.

Survey Note: Detailed Analysis of Using Makeblock Smart Camera with mBlock 5

This section provides a comprehensive exploration of the user’s queries regarding the Makeblock Smart Camera’s integration with mBlock 5, focusing on determining the number of detected colors and selecting which signal to use when multiple colors are detected. The analysis is grounded in available documentation, product descriptions, and related resources, particularly leveraging the Pixy2 camera technology, which underpins the Smart Camera.

Background and Context

The Makeblock Smart Camera is an intelligent vision sensor designed for robotics applications, such as color detection, line following, and barcode recognition. It is compatible with mBlock 5, a graphical programming platform based on Scratch, which simplifies coding for educational robotics. The camera can learn up to 6 different colors by pressing a “Learn” button and presenting objects of those colors, after which it can detect and report these colors as “blocks” in its field of view. The integration with mBlock 5 is facilitated through the “Smart Camera” extension, which provides blocks for programming and interacting with the camera’s capabilities.

Determining the Number of Colors Detected

To address the first query—how to know the number of colors detected—the analysis begins with understanding the camera’s detection mechanism. Documentation from Makeblock Education (Smart Camera | Makeblock Education) indicates that the Smart Camera can identify color blocks after learning and return their coordinates, length, and width, suggesting it tracks multiple objects simultaneously. Given its basis on Pixy technology, further insight is drawn from resources like the GitHub repository for mBlock-Pixy2 extension (GitHub - grantgunner/mblock-pixy2), which details blocks for similar cameras.

The key block identified is “get blocks,” which corresponds to the Arduino function pixy.ccc.getBlocks() and returns the count of detected blocks in the most recent frame. This block, part of the Smart Camera extension in mBlock 5, populates a data array for further access, effectively answering the query by providing the number of detected color objects. For instance, if the camera detects three red, blue, and green objects it was taught, “get blocks” would return 3.

The process involves:

  • Adding the “Smart Camera” extension in mBlock 5, accessible via the Extensions tab.
  • Using the “get blocks” block to query the current number of detected color blocks, which directly correlates to the number of colors detected, assuming each block represents a learned color.

This approach is supported by product descriptions, such as those from Robot-Advance (Smart Camera Makeblock for mBot and mBot 2), which mention the camera’s ability to distinguish up to 6 colors in color detection mode, reinforcing the functionality of the “get blocks” block.

Choosing Which Signal to Use When Multiple Colors Are Detected

The second query—how to choose which signal to use when the camera detects more than one color—requires handling multiple detected blocks and selecting one based on specific criteria. The analysis again leverages the Pixy2-based blocks, as detailed in the GitHub repository, which provides a table of mBlock blocks for interaction with the camera:

mBlock Block Arduino Code Parameters Description Notes
get blocks pixy.ccc.getBlocks() wait (default = true), sigmap (default = 255), maxblocks (default = 255) Gets all detected blocks in the most recent frame, returns count of blocks detected, populates data array for access. Parameters are optional, defaults pre-populated; numeric parameters casted to API datatype to avoid errors.
get block {variable} pixy.ccc.blocks[blocknum].{variable} e.g., pixy.ccc.blocks[0].m_signature blocknum - number of block to inspect (starts at 0) Returns specified attribute for a given block from the most recent getBlocks() call. Variables: signature (m_signature), x pos (m_x), y pos (m_y), width (m_width), height (m_height), angle (m_angle), index (m_index), age (m_age).
print block pixy.ccc.blocks[blocknum].print() blocknum - number of block to inspect Prints a summary of block variables to the console for debug purposes. -
set lamp pixy.setLamp() Top - on/off (controls top two white LEDs), Bottom - on/off (controls lower RGB LED, max for white light) Turns on/off Pixy2’s integrated light source. -
set LED pixy.setLED() Red component (0-255), Green component (0-255), Blue component (0-255) Sets the colour to be shown on the Pixy2 RGB LED. -
set servos pixy.setServos() pan position (0-1000), tilt position (0-1000) Controls the servo positions of the pan/tilt kit. Note: functionality not fully tested for position accuracy. Revision on 24-May-2019 changed to 16-bit value.

From this table, the “get block {variable}” block is crucial for accessing individual block details. Each detected block is indexed starting from 0, and variables such as signature (m_signature, corresponding to the learned color), x position (m_x), y position (m_y), width (m_width), and height (m_height) can be retrieved. This allows users to inspect each detected color’s properties and choose which one to use based on criteria like:

  • The largest block (comparing width and height).
  • A specific learned color (checking the signature, e.g., signature 1 for the first learned color).
  • Position (e.g., the block closest to the center of the camera’s view).

For example, if the camera detects two blocks—one with signature 1 (red) at position (100, 150) and another with signature 2 (blue) at (200, 200)—you can use “get block signature” with block number 0 to check if it’s red, and if so, use its position for further actions like robot movement.

This functionality is supported by documentation indicating the camera returns coordinates and size for each detected block (Smart Camera | Makeblock Education), and product pages like Jeulin (Smart camera Makeblock - Jeulin) that mention programming with mBlock 5 and the Smart Camera extension for such interactions.

Practical Implementation and Considerations

To implement these in mBlock 5:

  • Ensure the Smart Camera is properly connected to your robot (e.g., mBot or mBot2) and powered, preferably through the mBuild power module as recommended (Smart Camera | Makeblock Education).
  • Add the “Smart Camera” extension in mBlock 5, which should provide the blocks discussed, such as “get blocks” and “get block {variable}.”
  • For debugging, use the “print block” block to output details to the console, aiding in understanding which blocks are detected and their properties.

It’s worth noting that while the Pixy2-based blocks provide a detailed framework, the exact block names in mBlock 5 for the Smart Camera might vary slightly, but the functionality should align closely. For advanced users, the mBlock 5 Extension Builder can be used for custom blocks if needed, as mentioned in Makeblock documentation (Smart Camera | Makeblock Education).

Limitations and Further Resources

The analysis relies on Pixy2 documentation due to the lack of direct, accessible mBlock 5 Smart Camera extension documentation, which introduces some uncertainty. However, given the technological basis, the blocks and functionality are likely similar. Users are encouraged to consult the Makeblock Help Center (Makeblock Help Center) for official mBlock 5 guides and the mBlock Extension Center (mBlock Extension Center) for extension details. Additionally, forum discussions, such as those on the Makeblock Forum (Smart Camera code - mBlock - Makeblock Forum), may provide user insights and code examples.

Conclusion

In summary, to know the number of colors detected, use the “get blocks” block from the Smart Camera extension in mBlock 5, which returns the count of detected color objects. To choose which signal to use when multiple colors are detected, use the “get block {variable}” block to access each block’s details (signature, position, size) and select based on your criteria, ensuring flexible and effective programming for robotics applications.

Key Citations

#3

It’s not possible using block

#4

@FredRouv Yeah, I myself don’t have a smart camera, so I’m not sure. Looking at the extension, I do not see a block for it, maybe it can be done in the Python… I don’t know :confused: