[Discourse.ros.org] [ROS Projects] ROS Visual odometry

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

[Discourse.ros.org] [ROS Projects] ROS Visual odometry

Joshua Whitley via ros-users

# ROS Visual Odometry

# Contents
- Introduction
- System architecture
- Preparing the environment
- Calibrating the camera
- Rectifying image
- Getting odometry
- Visualizing pose
# **Introduction**

After this tutorial you will be able to create the system that determines position and orientation of a robot by analyzing the associated camera images. This information can be used in Simultaneous Localisation And Mapping (SLAM) problem that has been at the center of decades
of robotics research.


## What do we need to build this?
- Raspberry PI running Emlid Linux distro with pre-installed ROS
- Raspberry PI Camera
- Server running Linux

If you have everything, grab a cup of coffee, sit down, and read this tutorial.

# **System architecture**

If you absolutely have no idea what is ROS, nodes and how they communicate with each other, I strongly recommend you to learn it by reading official documentation and completing tutorials for beginners. Alternatively, you can read our guides. explaining its [basic architecture](https://docs.emlid.com/navio2/common/dev/ros/) and teaching how to write simple publisher and subscriber either on [Python](http://github.com/emlid/mavros-navio-python-example) or [C++](http://github.com/emlid/mavros-navio-cpp-example).
So, the graph of our system looks like this:


As you can see in this picture, we have Raspberry Camera connected and raspicam creating multimedia pipeline and sending video from camera to gst-launch. The latter then transmit the image to our server over UDP. gscam will broadcast the video to /raspicam/image_raw topic. This image should be rectified with image_proc node. And finally, rectified image is taken by mono_odometer, which handles it and computes position and orientation of the robot publishing this data straight to the /vision/pose topic.  

# **Preparing the environment**
## **Connecting the camera**

Firstly, connect your camera to Raspberry. To determine whether its working or not, just type:

    $ sudo vcgencmd get_camera

If you got **supported=1 detected=1**, ****then its ok and you can follow the next step. Otherwise, you should enable your camera with raspi-config. Furthermore, you can test video streaming with [this tutorial](https://docs.emlid.com/navio2/common/dev/video-streaming/).

## **Installing ROS on your server**

I used ROS kinetic, but you may use anything you want. Check out the [official guide](http://wiki.ros.org/kinetic/Installation) to get it working. Installation of ROS is quite straightforward and usually doesnt produce errors. If you stumbled and got any, you can always ask for help on the [forum](https://discourse.ros.org/).

## **Installing all packages and libraries**

In this section we are going to build our environment with every library we need. Here are the list of what we should install:

- OpenCV
- libviso2
- gscam
- image_common
- image_pipeline
- vision_opencv
## **OpenCV**

The needed packages should be installed using a terminal and the following commands:

    $ sudo apt-get install build-essential
    $ sudo apt-get install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev
    $ sudo apt-get install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev

Get the latest stable OpenCV version:

    $ git clone https://github.com/Itseez/opencv.git


    $ cd /opencv
    $ mkdir build
    $ cd build
    $ cmake -D CMAKE_BUILD_TYPE=Release -D CMAKE_INSTALL_PREFIX=/usr/local ..
    $ make -j5 # runs 5 jobs in parallel

All the following packages should be cloned into ~/odometry/src, so

    $ cd ~/odometry/src
## **viso2**
    $ git clone https://github.com/srv/viso2
## **gscam**
    $ git clone https://github.com/ros-drivers/gscam
## **image_common**
    $ git clone https://github.com/ros-perception/image_common.git
## **image_pipeline**
    $ git clone https://github.com/ros-perception/image_pipeline.git
## **vision_opencv**
    $ git clone https://github.com/ros-perception/vision_opencv.git

Building everything:

    $ cd ~/odometry
    $ catkin_make

After successful building all packages lets get our system up and working. Firstly, ssh into Raspberry and start broadcasting video to our server:

    $ raspivid -n -w 640 -h 480 -b 1000000 -fps 40 -t 0 -o - | gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay config-interval=10 pt=96 ! udpsink host=<IP> port=9000

 Where <IP> is IP address of your server. If you dont know it, type:

     $ ifconfig

and find your network and your IP.

After that change directory to ~/odometry/gscam/examples and create a new launch file called raspicam.launch:

    $ cd /odometry/gscam/examples
    $ vi raspicam.launch

Paste the following and save it:

       <arg name="cam_name" value="raspicam"/>
       <env name="GSCAM_CONFIG" value="udpsrc port=9000 ! application/x-rtp, payload=96 ! rtpjitterbuffer ! rtph264depay ! avdec_h264 ! videoconvert" />
       <node pkg="gscam" type="gscam" name="$(arg cam_name)">
         <param name="camera_name" value="$(arg cam_name)" />
         <remap from="camera/image_raw" to="$(arg cam_name)/image_raw" />

Then launch gscam and see if you can get an image:

    $ roslaunch gscam raspicam.launch
    $ rosrun image_view image_view image:=/raspicam/image_raw
# **Calibrating the camera**

Before Starting Make sure that you have a large [checkerboard](http://wiki.ros.org/camera_calibration/Tutorials/MonocularCalibration?action=AttachFile&do=view&target=check-108.pdf) with known dimensions.
To calibrate camera we will use cameracalibrator.py node from package image_calibration which is already installed. To run it for a monocular camera using an 8x6 chessboard with 24mm squares  just type:

    rosrun camera_calibration cameracalibrator.py --size 8x6 --square 0.024 image:=/raspicam/image_raw

You will see a new window opened which will highlight the checkerboard:. This will open up the calibration window:


In order to get a good calibration you will need to move the checkerboard around in the camera frame such that:

- checkerboard on the camera's left, right, top and bottom of field of view
  - X bar - left/right in field of view
  - Y bar - top/bottom in field of view
  - Size bar - toward/away and tilt from the camera
- checkerboard filling the whole field of view
- checkerboard tilted to the left, right, top and bottom

As you move the checkerboard around you will see three bars on the calibration sidebar increase in length. When the **CALIBRATE** button lights, you have enough data for calibration and can click **CALIBRATE** to see the results.


Calibration can take about a minute. The windows might be greyed out but just wait, it is working.
After calibration is done, you can save the archive and then extract it. You will need the *.yaml file. Rename it to raspicam.yaml and move it to the ~/odometry/src/gscam/example directory. Then open file raspicam.launch that weve already created and change it, so that it should looks like this:

       <arg name="cam_name" value="raspicam"/>
       <env name="GSCAM_CONFIG" value="udpsrc port=9000 ! application/x-rtp, payload=96 ! rtpjitterbuffer ! rtph264depay ! avdec_h264 ! videoconvert" />
       <node pkg="gscam" type="gscam" name="$(arg cam_name)">
         <param name="camera_name" value="$(arg cam_name)" />
         <param name="camera_info_url" value="package://gscam/examples/raspicam.yaml" />
         <remap from="camera/image_raw" to="$(arg cam_name)/image_raw" />

After that you have your camera calibrated and can launch gscam by:

    $ roslaunch gscam raspicam.launch

# **Rectifying the image**

The raw image from the camera driver is not what is needed for visual processing, but rather an undistorted and (if necessary) debayered image. This is the job of image_proc. For example, if you have topics /raspicam/image_raw and /raspicam/camera_info you would do:

    $ ROS_NAMESPACE=raspicam rosrun image_proc image_proc

There will appear a new topic /raspicam/image_rect. Its exactly what we need.

# **Getting odometry**
    $ rosrun viso2_ros mono_odometer image:=/raspicam/image_rect

This will publish /mono_odometer/pose messages and you can echo them:

    $ rostopic echo /mono_odometer/pose

# **Visualizing pose**

If you want to visualize that messages that is published into /mono_odometer/pose, then you should install and build another one package:

    $ cd ~/odometry/src
    $ git clone https://github.com/ros-visualization/rqt_pose_view.git 
    $ cd ~/odometry
    $ catkin_make

The **rqt_pose_view** is a very simple plugin just displaying an OpenGL 3D view showing a colored cube. You can drag and drop a geometry_msgs/Pose topic onto it from the "Topic Introspection" or "Publisher" plugins to make it visualize the orientation specified in the message.


[Visit Topic](https://discourse.ros.org/t/ros-visual-odometry/2465/1) or reply to this email to respond.

If you do not want to receive messages from ros-users please use the unsubscribe link below. If you use the one above, you will stop all of ros-users from receiving updates.
ros-users mailing list
[hidden email]
Unsubscribe: <http://lists.ros.org/mailman//options/ros-users>