Accessing the Cameras

Stretch 4 has four cameras in total. Three cameras are in the head, and one stereo-pair in the gripper.

This document will cover:

• Cameras Overview

  • Head cameras
  • Gripper camera
• Visualizing the cameras with ROS2

  • Using the cameras in your code with ROS2
• Visualizing the cameras with Python

  • Using the cameras in your code with python

Cameras Overview

Head Cameras

The two peripheral fisheye cameras named Left and Right, denoting their mount location relative to the robot's right and left, provide a wide field-of-view 1920x1200 resolution imagery at 30fps for seeing around the robot without rotating the base. The fisheye cameras can see up to 5ft behind Stretch 4, and all around the side and front of the robot.

The Center high-resolution 12MP camera is useful for reading small text or AI Object detection.

The Left and Right cameras can alternate to see below the arm. The center camera's view is susceptible to occlusion by the arm.

Gripper Camera

A stereo camera is mounted on Stretch 4's wrist, prior to the tool attachment site, to allow you to view objects in the robot's gripper or in front of the active tool. You can access the left and right camera feeds individually, if desired.

Visualizing The Cameras With ROS2

In your terminal, start these launch files:

Terminal 1:

ros2 launch stretch_core gripper_camera.launch.py use_rviz:=true

Terminal 2:

ros2 launch stretch_core luxonis.launch.py use_rviz:=true

The use_rviz parameter will start rviz2, with a live visualization of the cameras.

image

luxonis.launch.py and gripper_camera.launch.py are launch files that live on the robot at ~/ament_ws/src/stretch4_ros2/stretch_core/launch. Previews of these files can be found here.

Using The Cameras In Your Code With ROS2

The luxonis.launch.py launch file will publish three image_raw camera topics:

• /cameras_head/left/image_raw
• /cameras_head/center/image_raw
• /cameras_head/right/image_raw

The gripper_camera.launch.py launch file will publish two RGB camera topics, one stereo-depth camera topic, and one PointCloud2 stereo-depth topic:

• /cameras_gripper/left/image_raw
• /cameras_gripper/center/image_raw
• /cameras_gripper/stereo/image_raw
• /cameras_gripper/stereo_left_rgbd/points

You can subscribe to these topics in your ROS2 node to receive point cloud points.

Visualizing The Cameras With Python

To access the cameras using python, use stretch4_body.

In your terminal, launch:

stretch_camera_show --left_right_center --rerun

or

stretch_camera_show --gripper --rerun

This will start a Rerun window with the rotated images from the cameras.

Using The Cameras In Your Code With Python

stretch4_body offers a Stream API for accessing the camera streams using a python Generator:

from stretch4_body.subsystem.cameras import *

for image_frame in stream_left_camera():
    if image_frame is None: 
        print("No frame returned")
        continue
    print(f"Got image: {image_frame.image.shape=}, {image_frame.timestamp=}")

For more information, check out the Cameras Primer docs: https://github.com/hello-robot/stretch4_body/blob/main/docs/primer_cameras.md