[Discourse.ros.org] [Next Generation ROS] Latency and throughput in ROS2

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

[Discourse.ros.org] [Next Generation ROS] Latency and throughput in ROS2

Gary Edwards via ros-users


The paper [Maruyama, Yuya et al. Exploring the performance of ROS2. 2016 International Conference on Embedded Software (EMSOFT) (2016): 1-10.](https://www.semanticscholar.org/paper/Exploring-the-performance-of-ROS2-Maruyama-Kato/07b895f3b584dea4f64e91844f243de382026b20) evaluates the latency of ROS2 (different node configurations, different DDS implementations, different QoS policies, etc.).

Does someone know if these results have been updated by some more recent work(s)?





---
[Visit Topic](https://discourse.ros.org/t/latency-and-throughput-in-ros2/4367/1) or reply to this email to respond.


If you do not want to receive messages from ros-users please use the unsubscribe link below. If you use the one above, you will stop all of ros-users from receiving updates.
______________________________________________________________________________
ros-users mailing list
[hidden email]
http://lists.ros.org/mailman/listinfo/ros-users
Unsubscribe: <http://lists.ros.org/mailman//options/ros-users>
Reply | Threaded
Open this post in threaded view
|

[Discourse.ros.org] [Next Generation ROS] Latency and throughput in ROS2

Gary Edwards via ros-users


We will release a plugin-based tool that will let you evaluate latency and memory resource while altering number of pubs/subs, QoS, RT settings, large and small message types, publishing frequency, security settings, inter and intra process communication, etc. We need 2-3 more weeks, stay tuned.





---
[Visit Topic](https://discourse.ros.org/t/latency-and-throughput-in-ros2/4367/2) or reply to this email to respond.


If you do not want to receive messages from ros-users please use the unsubscribe link below. If you use the one above, you will stop all of ros-users from receiving updates.
______________________________________________________________________________
ros-users mailing list
[hidden email]
http://lists.ros.org/mailman/listinfo/ros-users
Unsubscribe: <http://lists.ros.org/mailman//options/ros-users>
Reply | Threaded
Open this post in threaded view
|

[Discourse.ros.org] [Next Generation ROS] Latency and throughput in ROS2

Gary Edwards via ros-users


That's fantastic @Dejan_Pangercic, looking forward to have a look.

On our side, we recently released a first tech report (https://arxiv.org/abs/1804.07643) that's part of a series that hopefully will characterize better the latencies and throughput in ROS 2 while taking in consideration Real-Time aspects. This first tech report treats OSI layers 1 and 2.





---
[Visit Topic](https://discourse.ros.org/t/latency-and-throughput-in-ros2/4367/3) or reply to this email to respond.


If you do not want to receive messages from ros-users please use the unsubscribe link below. If you use the one above, you will stop all of ros-users from receiving updates.
______________________________________________________________________________
ros-users mailing list
[hidden email]
http://lists.ros.org/mailman/listinfo/ros-users
Unsubscribe: <http://lists.ros.org/mailman//options/ros-users>
Reply | Threaded
Open this post in threaded view
|

[Discourse.ros.org] [Next Generation ROS] Latency and throughput in ROS2

Gary Edwards via ros-users
In reply to this post by Gary Edwards via ros-users


Hi all, we released the performance_test tool: https://github.com/ApexAI/performance_test to e.g. benchmark latency, jitter, lost samples, etc. in different DDS implementations.
Currently supported benchmarking is for communication mean over:
1. FastRTPS directly
1. ROS 2 rmw layer (and thus any supported rmw_* implementation)
1. Connext DDS Micro directly

We plan to extend it to use ROS1 comms as well.

Hope you find it useful, any feedaback is more than welcome.

D.





---
[Visit Topic](https://discourse.ros.org/t/latency-and-throughput-in-ros2/4367/4) or reply to this email to respond.


If you do not want to receive messages from ros-users please use the unsubscribe link below. If you use the one above, you will stop all of ros-users from receiving updates.
______________________________________________________________________________
ros-users mailing list
[hidden email]
http://lists.ros.org/mailman/listinfo/ros-users
Unsubscribe: <http://lists.ros.org/mailman//options/ros-users>
Reply | Threaded
Open this post in threaded view
|

[Discourse.ros.org] [Next Generation ROS] Latency and throughput in ROS2

Gary Edwards via ros-users
In reply to this post by Gary Edwards via ros-users


Hello !

Nice tool @Dejan_Pangercic !

What are the requirements on the RMW layer to be tested ?
I have a [non-DDS-based RMW](https://github.com/astralien3000/rmw_ndn), but it currently only support simple pub/sub.
It would be interesting to measure performance with your tool and compare it to DDS implementations.

Also (for latency), is it only measuring the end-to-end latency, or can it be more comprehensive ? For example measuring the time spent in each layer (RCL/RMW/DDS/Network).





---
[Visit Topic](https://discourse.ros.org/t/latency-and-throughput-in-ros2/4367/5) or reply to this email to respond.


If you do not want to receive messages from ros-users please use the unsubscribe link below. If you use the one above, you will stop all of ros-users from receiving updates.
______________________________________________________________________________
ros-users mailing list
[hidden email]
http://lists.ros.org/mailman/listinfo/ros-users
Unsubscribe: <http://lists.ros.org/mailman//options/ros-users>
Reply | Threaded
Open this post in threaded view
|

[Discourse.ros.org] [Next Generation ROS] Latency and throughput in ROS2

Gary Edwards via ros-users
In reply to this post by Gary Edwards via ros-users


Hi @astralien3000,

I am the maintainer of the performance test tool.

If you already have RMW implementation which supports pub/sub you should be able to directly test your communication mean without any additional work. You just need to set the proper environment variable `RMW_IMPLEMENTATION=rmw_ndn` before starting the tool.

It can not messure latency based on application layers unfortunately, for this it would have to be invasive in all these layers.

But you can create a new communication plugin for the NDN transport as I did for FastRTPS here:  https://github.com/ApexAI/performance_test/blob/master/performance_test/src/communication_abstractions/fast_rtps_communicator.hpp.

This will allow you to compare only the communication frameworks performance and will also give you some insight over the overhead the various RMW layers introduce.

If you run into issues implementing the plugin I will be glad to support you.





---
[Visit Topic](https://discourse.ros.org/t/latency-and-throughput-in-ros2/4367/6) or reply to this email to respond.


If you do not want to receive messages from ros-users please use the unsubscribe link below. If you use the one above, you will stop all of ros-users from receiving updates.
______________________________________________________________________________
ros-users mailing list
[hidden email]
http://lists.ros.org/mailman/listinfo/ros-users
Unsubscribe: <http://lists.ros.org/mailman//options/ros-users>
Reply | Threaded
Open this post in threaded view
|

[Discourse.ros.org] [Next Generation ROS] Latency and throughput in ROS2

Gary Edwards via ros-users
In reply to this post by Gary Edwards via ros-users


Thank you for your answer @andreaspasternak,

I was lacking some features (typesupport, proper management of multithreading, etc...), but in the end I was able to test my stack with your package. Also, I already implemented the invasive solution for measuring the latency of each layers. I will ask you if I need help, thank you again.





---
[Visit Topic](https://discourse.ros.org/t/latency-and-throughput-in-ros2/4367/7) or reply to this email to respond.


If you do not want to receive messages from ros-users please use the unsubscribe link below. If you use the one above, you will stop all of ros-users from receiving updates.
______________________________________________________________________________
ros-users mailing list
[hidden email]
http://lists.ros.org/mailman/listinfo/ros-users
Unsubscribe: <http://lists.ros.org/mailman//options/ros-users>
Reply | Threaded
Open this post in threaded view
|

[Discourse.ros.org] [Next Generation ROS] Latency and throughput in ROS2

Gary Edwards via ros-users


Hi @astralien3000,  
> Also, I already implemented the invasive solution for measuring the latency of each layers. I will ask you if I need help, thank you again.

Would you mind sharing some more details about your solution for measuring latency in each layer? Maybe we could even integrate it into the performance_test itself.





---
[Visit Topic](https://discourse.ros.org/t/latency-and-throughput-in-ros2/4367/8) or reply to this email to respond.


If you do not want to receive messages from ros-users please use the unsubscribe link below. If you use the one above, you will stop all of ros-users from receiving updates.
______________________________________________________________________________
ros-users mailing list
[hidden email]
http://lists.ros.org/mailman/listinfo/ros-users
Unsubscribe: <http://lists.ros.org/mailman//options/ros-users>
Reply | Threaded
Open this post in threaded view
|

[Discourse.ros.org] [Next Generation ROS] Latency and throughput in ROS2

Gary Edwards via ros-users
In reply to this post by Gary Edwards via ros-users


I didn't do anything complex, but since it is an invasive way of measuring, I don't know if it is easy to integrate into the performance_test. [I only printed events with their timestamp on stdio](https://github.com/astralien3000/rmw_fastrtps/blob/phd_meas_latency/rmw_fastrtps_cpp/src/rmw_publish.cpp#L53), and then post processed with a python script. It is not very precise, but since I don't want the real latency (only being able to compare the two implementations), it is ok for me. To avoid the print extra cost during the experiment, you can register the events+timestamp in a (pre-allocated) table, and print everything at the end.





---
[Visit Topic](https://discourse.ros.org/t/latency-and-throughput-in-ros2/4367/9) or reply to this email to respond.


If you do not want to receive messages from ros-users please use the unsubscribe link below. If you use the one above, you will stop all of ros-users from receiving updates.
______________________________________________________________________________
ros-users mailing list
[hidden email]
http://lists.ros.org/mailman/listinfo/ros-users
Unsubscribe: <http://lists.ros.org/mailman//options/ros-users>
Reply | Threaded
Open this post in threaded view
|

[Discourse.ros.org] [Next Generation ROS] Latency and throughput in ROS2

Gary Edwards via ros-users


Very nice. I was thinking if someone could do a  ros message extension to add-in the latencies at each layers in a private area of the message at each layer (encode as a BLOB) and could be retrieved at the subscriber to measure-in.





---
[Visit Topic](https://discourse.ros.org/t/latency-and-throughput-in-ros2/4367/10) or reply to this email to respond.


If you do not want to receive messages from ros-users please use the unsubscribe link below. If you use the one above, you will stop all of ros-users from receiving updates.
______________________________________________________________________________
ros-users mailing list
[hidden email]
http://lists.ros.org/mailman/listinfo/ros-users
Unsubscribe: <http://lists.ros.org/mailman//options/ros-users>