[Discourse.ros.org] [Computer Vision / Perception] Building a Tensorflow Object Detection and Localization ROS Package

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[Discourse.ros.org] [Computer Vision / Perception] Building a Tensorflow Object Detection and Localization ROS Package

ROS Agriculture via ros-users


Hey guys, longtime lurker, first time, you know.

I've been working on a project that involves custom building object D&L models and using them to find things in high-res streaming video. I have the model trainer pipeline done, and am building out the actual detection code using ROS (close to done). I'd like to turn this into a package for the community once it's done, but one thing I'm not clear on is dependency management. Tensorflow is in rapid development, and some of its dependencies may break other dependencies (numpy, etc.). Also, it's not a simple apt-get to install Tensorflow if you don't have CuDNN etc. installed first. Does anyone have any recommendations on how to manage these dependencies for a package?





---
[Visit Topic](https://discourse.ros.org/t/building-a-tensorflow-object-detection-and-localization-ros-package/3103/1) or reply to this email to respond.


If you do not want to receive messages from ros-users please use the unsubscribe link below. If you use the one above, you will stop all of ros-users from receiving updates.
______________________________________________________________________________
ros-users mailing list
[hidden email]
http://lists.ros.org/mailman/listinfo/ros-users
Unsubscribe: <http://lists.ros.org/mailman//options/ros-users>
Reply | Threaded
Open this post in threaded view
|

[Discourse.ros.org] [Computer Vision / Perception] Building a Tensorflow Object Detection and Localization ROS Package

ROS Agriculture via ros-users


Cannot waiting for the release. Since you mentioned it also does localization, may I ask how can you get the localization information (x, y and z) from a streaming video? Does it contain point cloud or something?





---
[Visit Topic](https://discourse.ros.org/t/building-a-tensorflow-object-detection-and-localization-ros-package/3103/2) or reply to this email to respond.


If you do not want to receive messages from ros-users please use the unsubscribe link below. If you use the one above, you will stop all of ros-users from receiving updates.
______________________________________________________________________________
ros-users mailing list
[hidden email]
http://lists.ros.org/mailman/listinfo/ros-users
Unsubscribe: <http://lists.ros.org/mailman//options/ros-users>
Reply | Threaded
Open this post in threaded view
|

[Discourse.ros.org] [Computer Vision / Perception] Building a Tensorflow Object Detection and Localization ROS Package

ROS Agriculture via ros-users
In reply to this post by ROS Agriculture via ros-users


I'm doing it on 2d images from a camera, with a depth sensor (tbd). No point clouds.





---
[Visit Topic](https://discourse.ros.org/t/building-a-tensorflow-object-detection-and-localization-ros-package/3103/3) or reply to this email to respond.


If you do not want to receive messages from ros-users please use the unsubscribe link below. If you use the one above, you will stop all of ros-users from receiving updates.
______________________________________________________________________________
ros-users mailing list
[hidden email]
http://lists.ros.org/mailman/listinfo/ros-users
Unsubscribe: <http://lists.ros.org/mailman//options/ros-users>
Reply | Threaded
Open this post in threaded view
|

[Discourse.ros.org] [Computer Vision / Perception] Building a Tensorflow Object Detection and Localization ROS Package

ROS Agriculture via ros-users
In reply to this post by ROS Agriculture via ros-users


so it would be a RGB-D sensor and because most of the well known RGB-D sensors' ROS wrapper provides registered point cloud data which also contains rgb field, I think you can also use point cloud data as the input.





---
[Visit Topic](https://discourse.ros.org/t/building-a-tensorflow-object-detection-and-localization-ros-package/3103/4) or reply to this email to respond.


If you do not want to receive messages from ros-users please use the unsubscribe link below. If you use the one above, you will stop all of ros-users from receiving updates.
______________________________________________________________________________
ros-users mailing list
[hidden email]
http://lists.ros.org/mailman/listinfo/ros-users
Unsubscribe: <http://lists.ros.org/mailman//options/ros-users>
Reply | Threaded
Open this post in threaded view
|

[Discourse.ros.org] [Computer Vision / Perception] Building a Tensorflow Object Detection and Localization ROS Package

ROS Agriculture via ros-users
In reply to this post by ROS Agriculture via ros-users


Any url to the code, or to a repository? I think it is an amazing project.

Best,
Paco





---
[Visit Topic](https://discourse.ros.org/t/building-a-tensorflow-object-detection-and-localization-ros-package/3103/5) or reply to this email to respond.


If you do not want to receive messages from ros-users please use the unsubscribe link below. If you use the one above, you will stop all of ros-users from receiving updates.
______________________________________________________________________________
ros-users mailing list
[hidden email]
http://lists.ros.org/mailman/listinfo/ros-users
Unsubscribe: <http://lists.ros.org/mailman//options/ros-users>
Reply | Threaded
Open this post in threaded view
|

[Discourse.ros.org] [Computer Vision / Perception] Building a Tensorflow Object Detection and Localization ROS Package

ROS Agriculture via ros-users
In reply to this post by ROS Agriculture via ros-users


Hi Drew! Glad to hear someone is working on this, it's something that I've felt ROS has needed for a while (and some partial implementations are out there).

Most ROS pipelines that I've seen that depend on TensorFlow, CuDNN, etc. being installed simply provide full instructions for installing the dependencies in their repository. Unfortunately this makes it hard to install via apt-get, and it's more complicated than cloning a git repository into your catkin workspace, but as long as the installation information is kept up to date, it's not too bad to copy-paste commands into a terminal.

I've gotten tensorflow and caffe interfaced with ROS before, and I had to install everything manually. I found the tutorials on PyImageSearch to be quite helpful, maybe you could reference those or make your own version that's more appropriate for your dependencies.

Also, I believe that tensorflow can be installed via `pip install tensorflow` even if you don't have CUDA installed, it will just install the CPU version of TensorFlow. So you could ask users to install tensorflow via pip, and provide links on how to install the dependencies if your users want more speed.

Also, it would be great to get people to start standarizing vision pipelines so they are interoperable. May I suggest that you make your detector output messages from the vision_msgs package?





---
[Visit Topic](https://discourse.ros.org/t/building-a-tensorflow-object-detection-and-localization-ros-package/3103/6) or reply to this email to respond.


If you do not want to receive messages from ros-users please use the unsubscribe link below. If you use the one above, you will stop all of ros-users from receiving updates.
______________________________________________________________________________
ros-users mailing list
[hidden email]
http://lists.ros.org/mailman/listinfo/ros-users
Unsubscribe: <http://lists.ros.org/mailman//options/ros-users>
Reply | Threaded
Open this post in threaded view
|

[Discourse.ros.org] [Computer Vision / Perception] Building a Tensorflow Object Detection and Localization ROS Package

ROS Agriculture via ros-users
In reply to this post by ROS Agriculture via ros-users


Paco - I'll clear some time out next week to clean up the code and put it out on github. I'll try to get it into a downloadable package for ROS too, if I can squeeze it in. I like to think there aren't that many dependencies and it should work fine, but I know better than to blindly believe that :slight_smile:

Kukanani - Pip can install tensorflow with or without GPU (assuming CuDNN, etc.). If you're doing other data sciencey stuff it can cause versioning / dependency issues by forcing some version upgrades (at least it's happened to me in the past, not sure how much of that still remains). For a lot of applications, I couldn't imagine not using GPU for Tensorflow. But then there's things like mobilenet and the Raspberry Pi, so it could be the case. Right now I'm favoring throwing an error about Tensorflow not being installed and point out the options, than having Tensorflow-CPU or -GPU be a dependency.

Thanks for recommending vision-msgs - I'll take a look. The current version of my code only returns pictures with bounding boxes, but returning probabilities / classifications / etc will be easy.





---
[Visit Topic](https://discourse.ros.org/t/building-a-tensorflow-object-detection-and-localization-ros-package/3103/7) or reply to this email to respond.


If you do not want to receive messages from ros-users please use the unsubscribe link below. If you use the one above, you will stop all of ros-users from receiving updates.
______________________________________________________________________________
ros-users mailing list
[hidden email]
http://lists.ros.org/mailman/listinfo/ros-users
Unsubscribe: <http://lists.ros.org/mailman//options/ros-users>