[Discourse.ros.org] [Next Generation ROS] Test Framework in ROS2

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[Discourse.ros.org] [Next Generation ROS] Test Framework in ROS2

Jonathan Binney via ros-users


I could not find any information about a "ROS node unit and integration" test framework in ROS2 (an equivalent to rostest in ROS1).

According to [Developer Guide - Testing (ros2 wiki)](https://github.com/ros2/ros2/wiki/Developer-Guide#testing) "Library unit testing (level 1)" is done with gtest, unittest, nosetest (mainline C++ and Python) [like in ROS1](http://wiki.ros.org/UnitTesting).

But what about "ROS node unit testing (level 2)" and "ROS node integration testing (level 3)"? In ROS1 people ended up to develop custom solutions to do ROS node unit and integration testing because the [restrictions and lack of useability of rostest](https://answers.ros.org/question/215600/how-can-i-run-roscore-from-python/?answer=215631#post-id-215631), [lack of generic tests nodes](https://answers.ros.org/question/264091/ros-node-unit-testing/), etc.

**BTW:** In ROS2 new [Package Categories](https://github.com/ros2/ros2/wiki/Developer-Guide#package-categories) are used to define requirements w.r.t. to testing specific to the "criticality" of ROS2 packages. I like that concept...





---
[Visit Topic](https://discourse.ros.org/t/test-framework-in-ros2/2667/1) or reply to this email to respond.


If you do not want to receive messages from ros-users please use the unsubscribe link below. If you use the one above, you will stop all of ros-users from receiving updates.
______________________________________________________________________________
ros-users mailing list
[hidden email]
http://lists.ros.org/mailman/listinfo/ros-users
Unsubscribe: <http://lists.ros.org/mailman//options/ros-users>
Reply | Threaded
Open this post in threaded view
|

[Discourse.ros.org] [Next Generation ROS] Test Framework in ROS2

Jonathan Binney via ros-users


[quote="fkromer, post:1, topic:2667"]
According to Developer Guide - Testing (ros2 wiki) Library unit testing (level 1) is done with gtest, unittest, nosetest (mainline C++ and Python) like in ROS1.
[/quote]

ROS 2 tries to be less opinionated than ROS 1. While `gtest`, `unittest`, and `nosetest` are supported (and heavily used to test ROS 2 code) they are not the only way. Integration with these tools is optional. E.g. ROS 2 also supports `gmock` (which ROS 1 doesn't do because of its difficult interaction with `gtest` when packaged separately). All of these testing frameworks are integrated using different ROS 2 packages. So adding support for more should be straight forward.

[quote="fkromer, post:1, topic:2667"]
But what about ROS node unit testing (level 2) and ROS node integration testing (level 3)? In ROS1 people ended up to develop custom solutions to do ROS node unit and integration testing because the restrictions and lack of useability of rostest, lack of generic tests nodes, etc.
[/quote]

At the moment we are using ROS 2 launch files as a replacement to an explicit `rostest` command. The package [launch_testing](https://github.com/ros2/launch/tree/master/launch_testing) provides a few different ways to check the result of a test node. This can be either a simple return code of one process (which decides on the result of the launch file and tears down all the other processes) or it could rely on specific output to be printed (matched by plain strings or regex). While the current syntax and API is everything but pretty it is functional and we use that to cover a lot of ROS 2 code. E.g. the `demo_nodes_cpp` providing simple talker, listener and services are tested using that. Since the same [launch file template](https://github.com/ros2/demos/blob/c6dd83c29115e056099dfc9fe3d48b35ca34e1d1/demo_nodes_cpp/test/test_executables_tutorial.py.in) is being used for multiple cases in this package it is a bit more complicated but it should provide an idea what is possible
 and how it works.





---
[Visit Topic](https://discourse.ros.org/t/test-framework-in-ros2/2667/2) or reply to this email to respond.


If you do not want to receive messages from ros-users please use the unsubscribe link below. If you use the one above, you will stop all of ros-users from receiving updates.
______________________________________________________________________________
ros-users mailing list
[hidden email]
http://lists.ros.org/mailman/listinfo/ros-users
Unsubscribe: <http://lists.ros.org/mailman//options/ros-users>
Reply | Threaded
Open this post in threaded view
|

[Discourse.ros.org] [Next Generation ROS] Test Framework in ROS2

Jonathan Binney via ros-users
In reply to this post by Jonathan Binney via ros-users


[quote="dirk-thomas, post:2, topic:2667"]
ROS 2 tries to be less opinionated than ROS 1.
[/quote]

[quote="dirk-thomas, post:2, topic:2667"]
E.g. ROS 2 also supports gmock (which ROS 1 doesnt do because of its difficult interaction with gtest when packaged separately).
[/quote]

That's great!

[quote="dirk-thomas, post:2, topic:2667"]
All of these testing frameworks are integrated using different ROS 2 packages. So adding support for more should be straight forward.
[/quote]

Is it possible to integrate other "coverage determination" frameworks than the usual gcov which provide more advanced coverage measurements as well?





---
[Visit Topic](https://discourse.ros.org/t/test-framework-in-ros2/2667/3) or reply to this email to respond.


If you do not want to receive messages from ros-users please use the unsubscribe link below. If you use the one above, you will stop all of ros-users from receiving updates.
______________________________________________________________________________
ros-users mailing list
[hidden email]
http://lists.ros.org/mailman/listinfo/ros-users
Unsubscribe: <http://lists.ros.org/mailman//options/ros-users>
Reply | Threaded
Open this post in threaded view
|

[Discourse.ros.org] [Next Generation ROS] Test Framework in ROS2

Jonathan Binney via ros-users
In reply to this post by Jonathan Binney via ros-users


[quote="fkromer, post:3, topic:2667"]
Is it possible to integrate other coverage determination frameworks than the usual gcov which provide more advanced coverage measurements as well?
[/quote]

You should be able to integrate any framework you like. I don't see where ROS 2 would prohibit you from doing so. ROS 2 is basically just a bunch of CMake packages - so anything you can do with a CMake package should be doable.





---
[Visit Topic](https://discourse.ros.org/t/test-framework-in-ros2/2667/4) or reply to this email to respond.


If you do not want to receive messages from ros-users please use the unsubscribe link below. If you use the one above, you will stop all of ros-users from receiving updates.
______________________________________________________________________________
ros-users mailing list
[hidden email]
http://lists.ros.org/mailman/listinfo/ros-users
Unsubscribe: <http://lists.ros.org/mailman//options/ros-users>