Repository navigation
Add Features to Align with Entity Topic Structure of MTConnect.NET Agent #535
Description
We are currently working with the MTConnect CPP Agent version 2.4.0.7, and are attempting to publish data via MQTT to an IoT Gateway, where the gateway is currently consuming MQTT data published by clients using the MTConnect.NET agent. However, the .NET agent publishes data with different payload structure and topic structure, which is preventing us from using the CPP Agent to publish data to this IoT Gateway. We would prefer not to switch to the .NET agent as that would force us to redo many of our build processes and re-test existing functionality, so I am opening this issue to request a few additions to the functionality offered by the CPP Agent.
The topic structure of the MTConnect.NET Agent that we want to emulate is:
ObservationTopic/<DataItemId>, whereObservationTopicis a configurable parameter likeCurrentTopic, ...- For example, if
ObservationTopicwasMTConnect/Devices/[device]/Observations, each data item would be published on the topicMTConnect/Devices/[device]/Observations/<DataItemId>
- For example, if
- See "Entity Topic Structure" in https://github.com/TrakHound/MTConnect.NET/tree/master/libraries/MTConnect.NET-MQTT for a more detailed description -- the .NET agent offers both a "Flat" format and "Hierarchy" format for Observation topics. We are asking for the flat format.
- The legacy MQTT sink in the CPP agent seemed to have functionality that published an Observation topic with a Hierarchy format. We need the flat format because the AWS IoT Core limits MQTT topics to 8 topics/7 forward slashes.
The payload structure for each data item that is used by the .NET Agent when publishing Observations is
{"dataItemId":"L2avail","name":"avail","type":"AVAILABILITY","timestamp":"2023-02-07T20:02:26.8978653Z","result":"AVAILABLE"}There have been some previous inquiries into having some of these features in the CPP Agent -- see the following:
- Query Regarding FlatMode Configuration in MTConnect CPP Agent
- Unified MQTT Topic and Payload structure
@wsobel Would it be possible to add this topic structure and payload format into the CPP Agent?
Activity
wsobel commentedon Apr 8, 2025
We deprecated the mqtt sink that is used per data item topics. We were having issues with broker performance, and it was dropping data. Maybe with Hive it will not be an issue.
This could also be added as a third-party plugin as well. I'm not sure what the .Net agent is doing. The MQTT support is not normative at this point; we need to make it normative in the future. There is support for Sparkplug B that is available from a third party.
We should discuss your requirements.
pacssys commentedon Apr 21, 2025
To shed a bit more light on our requirements, we are exploring multiple options to resolve the format differences between what the CPP agent publishes and what our Gateway is expecting. There are two main avenues we are exploring:
In either case, there is something I missed in the initial feature request:
Adding the "durable" buffer that the .NET agent has -- adding the capability to retain data when the agent restarts or loses connection with the broker. Upon reconnection, we would want to send all unsent data to the broker. This is something we would like to have eventually, but the foremost priority is resolving the difference in topic/payload structure.
wsobel commentedon Apr 24, 2025
There is a third option to create a plugin for the agent that will publish in the same format as the .NET agent. The agent has an extensible architecture and you can create a new MQTT sink that has the behavior you want. It does require some C++, but I can advise.
pacssys commentedon Jun 4, 2025
Thanks for your comments and sorry for the delayed response @wsobel -- we have been having internal discussions on the path we want to take.
We are still talking about how we want to approach this issue, but I wanted to gain some clarification regarding the third option to create a plugin to the agent.
To my understanding, creating a plugin to the agent would essentially be adding another sink to the agent (in addition to the rest_sink and mqtt_sink that currently exist) with the behavior we desire.
Is my understanding here correct?
wsobel commentedon Jun 4, 2025
Exactly. The sink has an interface and a fairly simple API. There is also a contract with the Agent to get various pieces of info like the observations and assets. The other items are posted. There is a timed callback mechanism that is used in the mqtt and the rest sinks.
The sink can then be used as a DLL or shared lib and dynamically loaded. This can all be orchestrated with Conan.
If you would like I can walk you through an example and maybe create some educational material in the process.
Best
pacssys commentedon Jun 4, 2025
That would be great! What would be easiest for you to use to walk through an example (Teams, this forum, ...)?
wsobel commentedon Nov 1, 2025
pacssys commentedon Nov 7, 2025
I see PR 571, which implements the requested feature, has been merged into main and is available in release v2.6.0.5.
Accordingly, I am closing this feature request.
Thanks to Will and Viraj!