[Discourse.ros.org] [Next Generation ROS] Increasing memory usage for publishing nodes with enabled intra-process communication

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[Discourse.ros.org] [Next Generation ROS] Increasing memory usage for publishing nodes with enabled intra-process communication

Mikael Arguedas via ros-users


I am looking into using ros2 and have done some testing on sending images between nodes using intraprocess communication. It seems like the memory usage is slowly increasing for all publishing nodes where intra-process communication is activated. This is regardless of any intra-process subscribers.

The same nodes with intra-process communication turned off does not have increasing memory usage.

I am currently using the default Fast-rtps implementation. I first noticed the problem in the "latest-release", but checked out the master branch yesterday to see if the problem was fixed. (it wasn't).

It can be reproduced by running the intra-process-demo image nodes.  

Is it possible to fix this?





---
[Visit Topic](https://discourse.ros.org/t/increasing-memory-usage-for-publishing-nodes-with-enabled-intra-process-communication/3261/1) or reply to this email to respond.


If you do not want to receive messages from ros-users please use the unsubscribe link below. If you use the one above, you will stop all of ros-users from receiving updates.
______________________________________________________________________________
ros-users mailing list
[hidden email]
http://lists.ros.org/mailman/listinfo/ros-users
Unsubscribe: <http://lists.ros.org/mailman//options/ros-users>
Reply | Threaded
Open this post in threaded view
|

[Discourse.ros.org] [Next Generation ROS] Increasing memory usage for publishing nodes with enabled intra-process communication

Mikael Arguedas via ros-users


Hi there,
    I just did a run on the latest master code.  There do seem to be a couple of leaks, so then I ran `image_pipeline_with_two_image_view` under valgrind (to be totally explicit, I ran `valgrind --leak-check=full ./build_isolated/intra_process_demo/image_pipeline_with_two_image_view`.  Looking at the output from that, it looks like the leaks are coming from somewhere down in the OpenCV code.  From a brief glance at it, it is not clear to me whether the leaks are in OpenCV itself, or how we are using it in the image demo.

All of that being said, this seems to be a problem with the interaction with OpenCV, not something fundamental to intra-process communication itself.  I probably won't have too much more time to spend on it right now, but if you have time to look at it, the code is in `ros2/demos/intra_process_demo/include/image_pipeline/image_view_node.hpp`





---
[Visit Topic](https://discourse.ros.org/t/increasing-memory-usage-for-publishing-nodes-with-enabled-intra-process-communication/3261/2) or reply to this email to respond.


If you do not want to receive messages from ros-users please use the unsubscribe link below. If you use the one above, you will stop all of ros-users from receiving updates.
______________________________________________________________________________
ros-users mailing list
[hidden email]
http://lists.ros.org/mailman/listinfo/ros-users
Unsubscribe: <http://lists.ros.org/mailman//options/ros-users>
Reply | Threaded
Open this post in threaded view
|

[Discourse.ros.org] [Next Generation ROS] Increasing memory usage for publishing nodes with enabled intra-process communication

Mikael Arguedas via ros-users
In reply to this post by Mikael Arguedas via ros-users


My program is actually not using OpenCV, and valgrind does not report memory leaks (even if the memory usage is increasing). However when I look at the memory usage in htop, the RES tab for my program it increases (quite slowly). I ran a test over the weekend, and the memory usage had continued to increase the whole weekend. When i turned off intra process communication and ran it over night, the RES tab showed no increase in memory usage...

I suspect that the pointers used in for the intra process communication are stored somewhere, or something like that.





---
[Visit Topic](https://discourse.ros.org/t/increasing-memory-usage-for-publishing-nodes-with-enabled-intra-process-communication/3261/3) or reply to this email to respond.


If you do not want to receive messages from ros-users please use the unsubscribe link below. If you use the one above, you will stop all of ros-users from receiving updates.
______________________________________________________________________________
ros-users mailing list
[hidden email]
http://lists.ros.org/mailman/listinfo/ros-users
Unsubscribe: <http://lists.ros.org/mailman//options/ros-users>
Reply | Threaded
Open this post in threaded view
|

[Discourse.ros.org] [Next Generation ROS] Increasing memory usage for publishing nodes with enabled intra-process communication

Mikael Arguedas via ros-users
In reply to this post by Mikael Arguedas via ros-users


@eirikhex can you provide code to reproduce this ?
I tried to reproduce the described behavior with a basic intra-process talker/listener and the reserved memory stayed steady (32124) after 20000 publications/subcriptions.





---
[Visit Topic](https://discourse.ros.org/t/increasing-memory-usage-for-publishing-nodes-with-enabled-intra-process-communication/3261/4) or reply to this email to respond.


If you do not want to receive messages from ros-users please use the unsubscribe link below. If you use the one above, you will stop all of ros-users from receiving updates.
______________________________________________________________________________
ros-users mailing list
[hidden email]
http://lists.ros.org/mailman/listinfo/ros-users
Unsubscribe: <http://lists.ros.org/mailman//options/ros-users>
Reply | Threaded
Open this post in threaded view
|

[Discourse.ros.org] [Next Generation ROS] Increasing memory usage for publishing nodes with enabled intra-process communication

Mikael Arguedas via ros-users
In reply to this post by Mikael Arguedas via ros-users


I also did a small test with a basic talker/listener. I used the "two_node_pipeline" from the intra process demo code:

> ros2/demos/intra_process_demo/include/two_node_pipeline/two_node_pipeline.cpp

The only change I made was to change the time interval in the timer on line 49. from 1s to 10ms to get results faster. I got the following results:
Starting memory                  21 420
after 80 000 publications:    28 100
after 100 000 publications:  29 948
after 175 000 publications:  37 724
after 200 000 publications:  39 308
And continues to rise.

I then changed both nodes by turning off intra-process communication, to verify that it in fact was something with the intra process communication causing the increased memory usage:
Starting memory:                  18036
after 200 000 publications:   18036

My original program sending a jpeg image over ros2 15 times per second has now been running for 6 days. It had a starting reserved memory of 33M, and has monotonically increased its reserved memory to 981M. I did the same test on the program with intra-process communication turned off and the memory did not increase.





---
[Visit Topic](https://discourse.ros.org/t/increasing-memory-usage-for-publishing-nodes-with-enabled-intra-process-communication/3261/5) or reply to this email to respond.


If you do not want to receive messages from ros-users please use the unsubscribe link below. If you use the one above, you will stop all of ros-users from receiving updates.
______________________________________________________________________________
ros-users mailing list
[hidden email]
http://lists.ros.org/mailman/listinfo/ros-users
Unsubscribe: <http://lists.ros.org/mailman//options/ros-users>
Reply | Threaded
Open this post in threaded view
|

[Discourse.ros.org] [Next Generation ROS] Increasing memory usage for publishing nodes with enabled intra-process communication

Mikael Arguedas via ros-users
In reply to this post by Mikael Arguedas via ros-users


Thanks @eirikhex for the additional details. I guess I didnt let it run for long then. I'll give it another shot





---
[Visit Topic](https://discourse.ros.org/t/increasing-memory-usage-for-publishing-nodes-with-enabled-intra-process-communication/3261/6) or reply to this email to respond.


If you do not want to receive messages from ros-users please use the unsubscribe link below. If you use the one above, you will stop all of ros-users from receiving updates.
______________________________________________________________________________
ros-users mailing list
[hidden email]
http://lists.ros.org/mailman/listinfo/ros-users
Unsubscribe: <http://lists.ros.org/mailman//options/ros-users>
Reply | Threaded
Open this post in threaded view
|

[Discourse.ros.org] [Next Generation ROS] Increasing memory usage for publishing nodes with enabled intra-process communication

Mikael Arguedas via ros-users
In reply to this post by Mikael Arguedas via ros-users


[quote="clalancette, post:2, topic:3261"]
I ran `valgrind --leak-check=full ./build_isolated/intra_process_demo/image_pipeline_with_two_image_view`.  Looking at the output from that, it looks like the leaks are coming from somewhere down in the OpenCV code.
[/quote]

The leaked memory from within `cvCreateMat` is constant - two times (one per window) independent how long the process is running. So this is not responsible for the increasing memory reported.





---
[Visit Topic](https://discourse.ros.org/t/increasing-memory-usage-for-publishing-nodes-with-enabled-intra-process-communication/3261/7) or reply to this email to respond.


If you do not want to receive messages from ros-users please use the unsubscribe link below. If you use the one above, you will stop all of ros-users from receiving updates.
______________________________________________________________________________
ros-users mailing list
[hidden email]
http://lists.ros.org/mailman/listinfo/ros-users
Unsubscribe: <http://lists.ros.org/mailman//options/ros-users>