Connecting to Stretch¶

In this guide, we will look at two ways of connecting to Stretch, tethered and untethered. Once connected, we'll run the Stretch Visual Servoing demo.

Tethered Setup¶

There is a computer inside Stretch called the "Intel NUC". You can develop and test your code directly on this computer.

In the robot's trunk, where the On/Off button is located, you will find several USB ports, a HDMI port, and an ethernet port. All of these are connected directly to the NUC inside the robot. There are additional USB ports located in the robot's head, wrist, and shoulder, which also connect to the NUC. This makes setting up a wired connection to the robot very easy.

The simplest setup for developing with Stretch is a monitor with an HDMI cable, and a USB keyboard/mouse. At Hello Robot, we prefer to use a wireless keyboard and mouse combo.

Simply plug the monitor into your robot using the included HDMI cable, and plug your keyboard and mouse into USB ports on the robot. Now turn on the robot again. That's it! After a few seconds, you'll see the NUC boot up sequence on the monitor.

Next, you'll see the Ubuntu desktop appear on the monitor. You can change your desktop background to a Stretch-themed wallpaper if you'd like.

The default user login credentials came in the box with the robot. By default, the robot is not configured to ask for your password on boot, but may ask for it later if the NUC goes to sleep. Stretch is running the Ubuntu 22.04 operating system.

Note

If you're using a Stretch 2 or Stretch RE1, your robot may be running an older operating system. See the troubleshooting advice below to identify which OS your robot is running.

Untethered Setup¶

As a mobile manipulator, the robot can only travel so far when tethered by the cables of a monitor, keyboard, and mouse. There are several ways to set up a wireless connection to Stretch, e.g. SSH, VNC, TeamViewer, AnyDesk, and in later tutorials we will explore some of these options and other programmatic methods of networking (e.g. ROS 2 networking, PyZMQ), but in this series we will be using a fast, open-source solution called "Moonlight".

Moonlight is software for streaming video games, but its ability to stream high-resolution, low-latency video makes it ideal for robotic applications as well. To set it up, follow these instructions:

You will need a "dummy HDMI dongle". If you're using a Stretch 3, one should be included in the accessories box. Otherwise, they can be found online for low cost.
You will need access to the robot's desktop. Go through the Wired Connection section if you haven't already.
On the robot, open Wifi settings and connect to a network. If you're using the Stretch WiFi Access Point, power up the router and Stretch will automatically connect to it.
On the robot, open a terminal and run:
```
systemctl start --user sunshine
```
Then, in the same terminal, run:
```
systemctl enable --user sunshine
```
Since this is your first time, we'll setup the Moonlight server (called "Sunshine") on Stretch. On the robot, open a browser and visit https://localhost:47990. You may see a warning and need to click "Advanced..." -> "Continue/Proceed". Next, it will ask you to choose credentials. Make sure to choose secure credentials and share them with anyone else using the robot. Then, log in and navigate to the "PIN" tab.
On your laptop or PC, go to moonlight-stream.org and download Moonlight.
Ensure your laptop or PC is connected to the same network as Stretch. Moonlight can stream over the internet, but it is configured to work on a local network by default.
Open Moonlight. It should automatically detect your Stretch. Click on it and you'll be given a 4 digit PIN. Enter this PIN in the Sunshine PIN page to pair Moonlight with your Stretch. Moonlight will remember this in the future.

Note

Sometimes Moonlight will not automatically detect the robot, even though both devices are on the same network. If this happens to you, see the troubleshooting advice below.
Moonlight will now show you a page with a button labeled "Desktop". This will start the stream of Stretch's desktop to your laptop or PC. Don't click on it yet. If you already did, exit out using Ctrl+Alt+Shift+Q.
Locate the dummy HDMI dongle included with Stretch. Disconnect all cables from Stretch, swapping the HDMI cable with the dummy HDMI dongle.
You can now click on the "Desktop" app in Moonlight to begin streaming the robot's desktop to your laptop/PC. You now have a wireless connection to Stretch!

To exit out of the stream, press Ctrl+Alt+Shift+Q. In the future, you can skip the pairing steps since the devices will already be paired.

Tips to optimize Moonlight¶

If the stream feels laggy, you can reduce the bitrate in Moonlight's settings.
If the mouse feels too fast, you can adjust the mouse behavior for remote desktop in Moonlight's settings.
If part of the Ubuntu desktop is cut off or the resolution is a poor match for your screen, you can adjust the display settings in Moonlight's settings.

Stretch Visual Servoing¶

If you've reached this point, you're connected to Stretch and ready to roll. Congrats! Let's try the Visual Servoing demo.

Visual servoing is a powerful way to achieve fast, precise, and robust motions. For this demo, your Stretch robot needs a Dex Wrist 3, which comes with an Intel Realsense D405 depth camera and a gripper with ArUco markers on the fingers.

Note

If you're using a Stretch 2 or Stretch RE1, your robot may not have a Dex Wrist 3. See the troubleshooting advice below to identify which tool your robot has.

To start the demo, follow these instructions:

Clone this repository on your robot

git clone https://github.com/hello-robot/stretch_visual_servoing/

Next, find the transparent acrylic cube that came with your robot. It has a 30 mm by 30 mm ArUco marker on it that encodes the ArUco ID# 202. The robot will try to reach for and grab this cube when it sees the ArUco marker with the D405 camera on its gripper.
Home the robot
```
stretch_robot_home.py
```
You may be asked to run stretch_free_robot_process.py before homing.
Enter into the home directory of the cloned repository
```
cd ./stretch_visual_servoing
```
Run the visual servoing demo
```
python3 visual_servoing_demo.py
```

The Visual Servoing demo also works with a tennis ball. Check out the instructions in the repo to try it out.

Next Steps¶

In the next guide, Stretch Hardware Overview, we will start to get familiar with the robot hardware and some common command line tools.

Troubleshooting¶

If you're having trouble with the steps in the guide, please check the following tips:

Which operating system is my robot running?¶

Open system setting by clicking "Settings" in the control panel
Scroll down the "About" page
Locate the "OS Name". Stretch has had 3 OS releases: 18.04, 20.04, and 22.04

Ubuntu 22.04 is the latest at the time of this writing. The tutorials on this website have been written with 22.04 in mind. If your Stretch is running an older version, consider upgrading using the Upgrading your Operating System guide.

Stretch isn't automatically detected by Moonlight¶

Some routers block local networking. You can try adding the robot by entering it's IP address into Moonlight. On the robot, open a terminal and enter ifconfig. The output typically looks like this. You can find the robot's IP address by searching for the address near terms like "wlp2s0" and "inet" (e.g. 192.168.1.29 in the image below).
Some routers block all local networking, so it isn't possible to use Moonlight. If possible, use a different router.

Does my robot have a Dex Wrist 3?¶

Here's what the Dex Wrist 3 looks like.
If your robot doesn't have a Dex Wrist 3, you can skip the Visual Servoing section.

Connect to your robot using the tethered setup. Open a terminal and run the following command:

# Choose a new username and password
sunshine --creds <new-username> <new-password>