[Discourse.ros.org] [Next Generation ROS] ROS2: anything equivalent to topic_tools:ShapeShifter?
No, unfortunately we don't have that yet. @karsten was working on it for this release, but we ran into some issues figuring out how to most efficiently get data in and out of the underlying DDS vendors without (de)serializing. The API we'll provide will be relatively simple, something like `rmw_publish_raw` and `rmw_take_raw` and some wrapper like `ShapeShifter` in our C++ API, but it's the "how" that's held us up.
But we're still looking at this issue and it's in our critical path as we need it to implement rosbag and to implement "type masquerading".
I'll try to remember to reply here when we get the first version of that new API figured out.
The other thing I suppose you'll need is a way to parse the "ROS IDL" files, e.g. `.msg`? We don't currently have that in anything but Python. I see that you have one for ROS 1 in C++:
Also, it would be awesome to have a separate library for parsing the various interface files in C/C++, so it could be reused everywhere.
There's also the possibility to have another, more machine friendly, version of the interfaces. For example, as part of generation at build time we could generate an xml or json file which represents the contents of the corresponding `.msg` file. Then you could lookup and load that file instead (with an xml or json library), which would save you from needing to writing parsing code, and instead you would only need to interpret the contents correctly.
This is a feature I'd really like to see well supported in both ROS 1 and ROS 2, it would be especially helpful in making the bridge between the two more convenient to use with non-standard types. So, once we get you unblocked by providing something like ShapeShifter, I'd be really interested in helping you with anything you're missing to implement it.
The "machine friendly" format you mentioned would (of course) be VERY welcome but not strictly necessary, since I have done already most of the heavy lifting in terms of parsing.
One quick question. My runtime deserialization was created doing a "reverse engineering" of how ROS1 messages are created.
In DDS is the serialization protocol standardized or middleware dependent? Since DDS is known to have good interoperability, I am tempted to think that the former is true...
The DDS serialization protocol is indeed standardized. It uses a format called Extended Common Data Representation (XCDR). The most up-to-date reference is the DDS-XTYPES specification version 1.2. See http://www.omg.org/spec/DDS-XTypes/
The implementation of this imagined `deserialize_message` function may end up calling an implementation specific function, or at the very least how it is deserialized should be an implementation detail.
We have the beginnings of this in https://github.com/ros2/rosidl/tree/master/rosidl_typesupport_introspection_cpp, but I'm 100% certain how much is still needed to support this at the user level. We might not be able to use any of that package at the user level, since it isn't always used in the implementation, but it is at least representing the structure of a message as C++ structs and objects which is part of what we'd need to do. My initial guess is that we would need to have a way to generate the same typesupport structures but given an input string or structure which describes the type.
So I'd actually say we should try to avoid your package needing to reverse engineer how to deserialize, instead that should be provided by our abstraction layer. This allows us to protect the stuff that comes on top from underlying changes, e.g. if we changed from one version of CDR (the serialization standard used in DDS and ROS 2) to another.
- The roadmap currently contains a task to revisit the message definition format. The `.msg` format currently used with some minor extensions from ROS 1 might be replaced with something more powerful since the format can't handle several of the pending feature requests. Just to be consider when spending effort on implementing the current format in different languages.
- While the serialization protocol of the DDS implementations follows a single standard you can't rely on the rmw impl. to be DDS. There has e.g. be efforts to implement the rmw interface with other middlewares like OPC UA.
1) Type erased messages are needed for applications such as rosbag, rqt_plot, PlotJuggler, MATLAB importers, etc. Any generic topic subscriber indeed.
2) ShapeShifter had all the informations needed to do this: raw bytes buffer, Message definition and MD5Sum.
3) ROS1 had just one protocol, whilst ROS2 can potentially have DDS or others.
THINGS WE MIGHT DO:
A) I guess that the first step is to add to ROSIDL the equivalent of __ros::message_traits::Definition< Type >::value()__. Preferably expressed as JSON or XML format.
B) It must be possible to subscribe to a topic in a "generic" way, i.e. bypassing the deserialization step and accessign the raw bytes of the message.
C) To implement a run-time deserializer, I don't see any solution but implementing a different code for each protocol :frowning: No magic wands.
[quote="facontidavide, post:7, topic:3393"]
A) I guess that the first step is to add to ROSIDL the equivalent of ros::message_traits::Definition< Type >::value(). Preferably expressed as JSON or XML format.
I assume the value should contain the message description in a readable format. Why should that be provided through a language specific symbol? I would suggest to provide this information through the ament resource index which would make it available across languages.